This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 5a02fae9ef24a26627a32987ea738df5ea858929 (commit) via 0841f2bd47d6f0134c4881e9b9e3f3dba3404f55 (commit) from 398f4af12cf6d05176886515c8f42df870c92631 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/5a02fae9ef24a26627a32987ea738df5ea... commit 5a02fae9ef24a26627a32987ea738df5ea858929 Author: Ken Raffenetti <[email protected]> Date: Fri Aug 26 10:07:41 2016 -0500 maint/version.m4: Update version info and CHANGES Prep for 3.3a1 release. Signed-off-by: Pavan Balaji <[email protected]> diff --git a/CHANGES b/CHANGES index ca19721..fd3c771 100644 --- a/CHANGES +++ b/CHANGES @@ -1,4 +1,26 @@ =============================================================================== + Changes in 3.3 +=============================================================================== + + # CH4 Device: A new device layer implementation designed for low software + overheads. CH4 has experimental support for OFI and UCX network libraries, + and POSIX shared memory. Thanks to Intel, Mellanox, and RIKEN AICS for + participating in the CH4 coding effort. + + # Removed the PAMI device and poe PMI client. + + # Several other minor bug fixes, memory leak fixes, and code cleanup. + + A full list of changes is available at the following link: + + http://git.mpich.org/mpich.git/shortlog/v3.2..v3.3a1 + + A full list of bugs that have been fixed is available at the + following link: + + https://trac.mpich.org/projects/mpich/query?status=closed&group=resolution&m... + +=============================================================================== Changes in 3.2 =============================================================================== diff --git a/maint/version.m4 b/maint/version.m4 index d3e271e..7957dbd 100644 --- a/maint/version.m4 +++ b/maint/version.m4 @@ -14,7 +14,7 @@ # changing this by playing with diversions, but then we would probably be # playing with autotools-fire. -m4_define([MPICH_VERSION_m4],[3.2])dnl +m4_define([MPICH_VERSION_m4],[3.3a1])dnl m4_define([MPICH_RELEASE_DATE_m4],[unreleased development copy])dnl # For libtool ABI versioning rules see: @@ -35,6 +35,6 @@ m4_define([MPICH_RELEASE_DATE_m4],[unreleased development copy])dnl # libmpi so version only includes functionality defined in the MPI # standard, and does not include MPIX_ functions and C++ bindings. -m4_define([libmpi_so_version_m4],[13:0:1])dnl +m4_define([libmpi_so_version_m4],[0:0:0])dnl [#] end of __file__ http://git.mpich.org/mpich.git/commitdiff/0841f2bd47d6f0134c4881e9b9e3f3dba3... commit 0841f2bd47d6f0134c4881e9b9e3f3dba3404f55 Author: Ken Raffenetti <[email protected]> Date: Fri Aug 26 10:31:18 2016 -0500 mpid: Remove PAMI device Remove unsupported pamid device, including poe PMI client. Signed-off-by: Pavan Balaji <[email protected]> diff --git a/README.vin b/README.vin index 40b6e72..c845b17 100644 --- a/README.vin +++ b/README.vin @@ -577,33 +577,6 @@ channel can be configured using the following option: --with-device=ch3:sock - -pamid device -************ -This is the device used on the IBM Blue Gene/Q system. The following -configure options can be used: - - ./configure --host=powerpc64-bgq-linux \ - --with-device=pamid:BGQ \ - --with-file-system=bg+bglockless - -The Blue Gene/Q cross compilers must either be in the $PATH, or -explicitly specified using environment variables, before configure. -For example: - - PATH=$PATH:/bgsys/drivers/ppcfloor/gnu-linux/bin - -or - - CC=/bgsys/drivers/ppcfloor/gnu-linux/bin/powerpc64-bgq-linux-gcc - CXX=... - ... - -There are several other configure options that are specific to building -on a Blue Gene/Q system. See the wiki page for more information: - - https://wiki.mpich.org/mpich/index.php/BGQ - ------------------------------------------------------------------------- 5. Alternate Process Managers diff --git a/src/mpid/Makefile.mk b/src/mpid/Makefile.mk index 84122ef..247a485 100644 --- a/src/mpid/Makefile.mk +++ b/src/mpid/Makefile.mk @@ -13,5 +13,4 @@ noinst_HEADERS += \ include $(top_srcdir)/src/mpid/ch3/Makefile.mk include $(top_srcdir)/src/mpid/ch4/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/Makefile.mk include $(top_srcdir)/src/mpid/common/Makefile.mk diff --git a/src/mpid/pamid/.gitattributes b/src/mpid/pamid/.gitattributes deleted file mode 100644 index 3d45ef0..0000000 --- a/src/mpid/pamid/.gitattributes +++ /dev/null @@ -1,15 +0,0 @@ -* whitespace=trail-space,space-before-tab -*.[ch] whitespace=trail-space,space-before-tab,cr-at-eol,tab-in-indent -Makefile.sm whitespace=trail-space,space-before-tab,cr-at-eol - - -# * `blank-at-eof` treats blank lines added at the end of file as an error (enabled by default). -# * `blank-at-eol` treats trailing whitespaces at the end of the line as an error (enabled by default). -# * `trailing-space` is a short-hand to cover both `blank-at-eol` and `blank-at-eof`. -# * `space-before-tab` treats a space character that appears immediately before a tab character in the initial indent part of the line as an error (enabled by default). -# * `cr-at-eol` treats a carriage-return at the end of line as part of the line terminator, i.e. with it, `trailing-space` does not trigger if the character before such a carriage-return is not a whitespace (not enabled by default). - -# * `indent-with-non-tab` treats a line that is indented with 8 or more space characters as an error (not enabled by default). -# * `tab-in-indent` treats a tab character in the initial indent part of the line as an error (not enabled by default). - -# * `tabwidth=<n>` tells how many character positions a tab occupies; this is relevant for `indent-with-non-tab` and when git fixes `tab-in-indent` errors. The default tab width is 8. Allowed values are 1 to 63. diff --git a/src/mpid/pamid/.gitignore b/src/mpid/pamid/.gitignore deleted file mode 100644 index f6f157a..0000000 --- a/src/mpid/pamid/.gitignore +++ /dev/null @@ -1,3 +0,0 @@ -/autom4te.cache -/configure -Makefile.in diff --git a/src/mpid/pamid/Makefile.mk b/src/mpid/pamid/Makefile.mk deleted file mode 100644 index f33946c..0000000 --- a/src/mpid/pamid/Makefile.mk +++ /dev/null @@ -1,43 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - -noinst_HEADERS += \ - src/mpid/pamid/include/mpidi_thread.h \ - src/mpid/pamid/include/mpidimpl.h \ - src/mpid/pamid/include/mpidi_mutex.h \ - src/mpid/pamid/include/mpidpost.h \ - src/mpid/pamid/include/mpidi_constants.h \ - src/mpid/pamid/include/mpidi_externs.h \ - src/mpid/pamid/include/mpidi_util.h \ - src/mpid/pamid/include/mpidi_hooks.h \ - src/mpid/pamid/include/mpidi_macros.h \ - src/mpid/pamid/include/mpidi_datatypes.h \ - src/mpid/pamid/include/mpidi_prototypes.h \ - src/mpid/pamid/include/pamix.h \ - src/mpid/pamid/include/mpidpre.h \ - src/mpid/pamid/include/mpidi_platform.h - -include_HEADERS += src/mpid/pamid/include/mpix.h - -include $(top_srcdir)/src/mpid/pamid/src/Makefile.mk - -endif BUILD_PAMID diff --git a/src/mpid/pamid/cross/bgq8 b/src/mpid/pamid/cross/bgq8 deleted file mode 100644 index 7cecbb1..0000000 --- a/src/mpid/pamid/cross/bgq8 +++ /dev/null @@ -1,27 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -CROSS_F77_SIZEOF_INTEGER=4 -CROSS_F77_SIZEOF_REAL=4 -CROSS_F77_SIZEOF_DOUBLE_PRECISION=8 -CROSS_F90_ADDRESS_KIND=8 -CROSS_F90_OFFSET_KIND=8 -CROSS_F90_INTEGER_KIND=8 -CROSS_F90_REAL_MODEL=6,37 -CROSS_F90_DOUBLE_MODEL=15,307 -CROSS_F90_INTEGER_MODEL_MAP={9,4,4}, -CROSS_F77_TRUE_VALUE=1 -CROSS_F77_FALSE_VALUE=0 diff --git a/src/mpid/pamid/cross/pe4 b/src/mpid/pamid/cross/pe4 deleted file mode 100644 index 2fc1d9e..0000000 --- a/src/mpid/pamid/cross/pe4 +++ /dev/null @@ -1,27 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -CROSS_F77_SIZEOF_INTEGER=4 -CROSS_F77_SIZEOF_REAL=4 -CROSS_F77_SIZEOF_DOUBLE_PRECISION=8 -CROSS_F90_ADDRESS_KIND=4 -CROSS_F90_OFFSET_KIND=8 -CROSS_F90_INTEGER_KIND=4 -CROSS_F90_REAL_MODEL=6,37 -CROSS_F90_DOUBLE_MODEL=15,307 -CROSS_F90_INTEGER_MODEL_MAP={9,4,4}, -CROSS_F77_TRUE_VALUE=1 -CROSS_F77_FALSE_VALUE=0 \ No newline at end of file diff --git a/src/mpid/pamid/cross/pe4-aix b/src/mpid/pamid/cross/pe4-aix deleted file mode 100644 index f3478d1..0000000 --- a/src/mpid/pamid/cross/pe4-aix +++ /dev/null @@ -1,28 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -CROSS_F77_SIZEOF_INTEGER=4 -CROSS_F77_SIZEOF_REAL=4 -CROSS_F77_SIZEOF_DOUBLE_PRECISION=8 -CROSS_F90_ADDRESS_KIND=4 -CROSS_F90_OFFSET_KIND=8 -CROSS_F90_INTEGER_KIND=4 -CROSS_F90_REAL_MODEL=6,37 -CROSS_F90_DOUBLE_MODEL=15,307 -CROSS_F90_INTEGER_MODEL_MAP={9,4,4}, -CROSS_F77_TRUE_VALUE=1 -CROSS_F77_FALSE_VALUE=0 -CROSS_ALIGN_DOUBLE_EXCEPTION="four" diff --git a/src/mpid/pamid/cross/pe8 b/src/mpid/pamid/cross/pe8 deleted file mode 100644 index 866fdd1..0000000 --- a/src/mpid/pamid/cross/pe8 +++ /dev/null @@ -1,27 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -CROSS_F77_SIZEOF_INTEGER=4 -CROSS_F77_SIZEOF_REAL=4 -CROSS_F77_SIZEOF_DOUBLE_PRECISION=8 -CROSS_F90_ADDRESS_KIND=8 -CROSS_F90_OFFSET_KIND=8 -CROSS_F90_INTEGER_KIND=8 -CROSS_F90_REAL_MODEL=6,37 -CROSS_F90_DOUBLE_MODEL=15,307 -CROSS_F90_INTEGER_MODEL_MAP={9,4,4}, -CROSS_F77_TRUE_VALUE=1 -CROSS_F77_FALSE_VALUE=0 \ No newline at end of file diff --git a/src/mpid/pamid/cross/pe8-aix b/src/mpid/pamid/cross/pe8-aix deleted file mode 100644 index 8380c89..0000000 --- a/src/mpid/pamid/cross/pe8-aix +++ /dev/null @@ -1,28 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -CROSS_F77_SIZEOF_INTEGER=4 -CROSS_F77_SIZEOF_REAL=4 -CROSS_F77_SIZEOF_DOUBLE_PRECISION=8 -CROSS_F90_ADDRESS_KIND=8 -CROSS_F90_OFFSET_KIND=8 -CROSS_F90_INTEGER_KIND=8 -CROSS_F90_REAL_MODEL=6,37 -CROSS_F90_DOUBLE_MODEL=15,307 -CROSS_F90_INTEGER_MODEL_MAP={9,4,4}, -CROSS_F77_TRUE_VALUE=1 -CROSS_F77_FALSE_VALUE=0 -CROSS_ALIGN_DOUBLE_EXCEPTION="four" diff --git a/src/mpid/pamid/include/.gitignore b/src/mpid/pamid/include/.gitignore deleted file mode 100644 index ea8dee9..0000000 --- a/src/mpid/pamid/include/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/mpid_config.h.in diff --git a/src/mpid/pamid/include/mpidi_constants.h b/src/mpid/pamid/include/mpidi_constants.h deleted file mode 100644 index 25c7c5a..0000000 --- a/src/mpid/pamid/include/mpidi_constants.h +++ /dev/null @@ -1,128 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_constants.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_constants_h__ -#define __include_mpidi_constants_h__ - -enum - { - /* N is "number of operations" and P is number of ranks */ - MPIDI_VERBOSE_NONE = 0, /**< Do not print any verbose information */ - MPIDI_VERBOSE_SUMMARY_0 = 1, /**< Print summary information on rank 0. O(1) lines printed. */ - MPIDI_VERBOSE_SUMMARY_ALL = 2, /**< Print summary information on all ranks. O(P) lines printed. */ - MPIDI_VERBOSE_DETAILS_0 = 2, /**< Print detailed information on rank 0. O(N) lines printed. */ - MPIDI_VERBOSE_DETAILS_ALL = 3, /**< Print detailed information on all ranks. O(P*N) lines printed. */ - }; - - -/** - * \defgroup Allgather(v) optimization datatype info - * \{ - */ -enum - { - MPID_SEND_CONTIG = 0, /**< Contiguous send buffer */ - MPID_RECV_CONTIG = 1, /**< Contiguous recv buffer */ - MPID_RECV_CONTINUOUS = 2, /**< Continuous recv buffer */ - MPID_LARGECOUNT = 3, /**< Total send count is "large" */ - MPID_MEDIUMCOUNT = 4, /**< Total send count is "medium" */ - MPID_ALIGNEDBUFFER = 5, /**< Buffers are 16b aligned */ - }; - -enum - { - MPID_ALLGATHER_PREALLREDUCE = 0, - MPID_ALLGATHERV_PREALLREDUCE = 1, - MPID_ALLREDUCE_PREALLREDUCE = 2, - MPID_BCAST_PREALLREDUCE = 3, - MPID_SCATTERV_PREALLREDUCE = 4, - MPID_GATHER_PREALLREDUCE = 5, - MPID_NUM_PREALLREDUCES = 6, - }; - -enum /* The type of protocol selected */ - { - MPID_COLL_NOQUERY = 0, - MPID_COLL_QUERY = 1, - /* Can we cache stuff? If not set to ALWAYS_QUERY */ - MPID_COLL_ALWAYS_QUERY = 2, - MPID_COLL_CHECK_FN_REQUIRED = 3, - MPID_COLL_USE_MPICH = 4, - MPID_COLL_NOSELECTION = 5, - MPID_COLL_OPTIMIZED = 6, - }; - -enum - { - MPID_COLL_OFF = 0, - MPID_COLL_ON = 1, - MPID_COLL_FCA = 2, /* Selecting these is fairly easy so special case */ - MPID_COLL_CUDA = 3, /* This is used to enable PAMI geometry but sets default to MPICH */ - }; -/** \} */ - - -enum -{ -MPID_EPOTYPE_NONE = 0, /**< No epoch in affect */ -MPID_EPOTYPE_LOCK = 1, /**< MPI_Win_lock access epoch */ -MPID_EPOTYPE_START = 2, /**< MPI_Win_start access epoch */ -MPID_EPOTYPE_POST = 3, /**< MPI_Win_post exposure epoch */ -MPID_EPOTYPE_FENCE = 4, /**< MPI_Win_fence access/exposure epoch */ -MPID_EPOTYPE_REFENCE = 5, /**< MPI_Win_fence possible access/exposure epoch */ -MPID_EPOTYPE_LOCK_ALL = 6, /**< MPI_Win_lock_all access epoch */ -}; - -enum -{ - MPID_AUTO_SELECT_COLLS_NONE = 0, - MPID_AUTO_SELECT_COLLS_BARRIER = 1, - MPID_AUTO_SELECT_COLLS_BCAST = ((int)((MPID_AUTO_SELECT_COLLS_BARRIER << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_ALLGATHER = ((int)((MPID_AUTO_SELECT_COLLS_BCAST << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_ALLGATHERV = ((int)((MPID_AUTO_SELECT_COLLS_ALLGATHER << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_ALLREDUCE = ((int)((MPID_AUTO_SELECT_COLLS_ALLGATHERV << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_ALLTOALL = ((int)((MPID_AUTO_SELECT_COLLS_ALLREDUCE << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_ALLTOALLV = ((int)((MPID_AUTO_SELECT_COLLS_ALLTOALL << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_EXSCAN = ((int)((MPID_AUTO_SELECT_COLLS_ALLTOALLV << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_GATHER = ((int)((MPID_AUTO_SELECT_COLLS_EXSCAN << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_GATHERV = ((int)((MPID_AUTO_SELECT_COLLS_GATHER << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_REDUCE_SCATTER = ((int)((MPID_AUTO_SELECT_COLLS_GATHERV << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_REDUCE = ((int)((MPID_AUTO_SELECT_COLLS_REDUCE_SCATTER << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_SCAN = ((int)((MPID_AUTO_SELECT_COLLS_REDUCE << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_SCATTER = ((int)((MPID_AUTO_SELECT_COLLS_SCAN << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_SCATTERV = ((int)((MPID_AUTO_SELECT_COLLS_SCATTER << 1) & 0xFFFFFFFF)), - MPID_AUTO_SELECT_COLLS_TUNE = 0x80000000, - MPID_AUTO_SELECT_COLLS_ALL = 0xFFFFFFFF, -}; - -enum /* PAMID_COLLECTIVES_MEMORY_OPTIMIZED levels */ - -{ - MPIR_OPT_LVL_IRREG = 1, /**< Do not optimize irregular communicators */ - MPIR_OPT_LVL_NONCONTIG = 2, /**< Disable some non-contig collectives */ -}; -#endif diff --git a/src/mpid/pamid/include/mpidi_datatypes.h b/src/mpid/pamid/include/mpidi_datatypes.h deleted file mode 100644 index b716fc9..0000000 --- a/src/mpid/pamid/include/mpidi_datatypes.h +++ /dev/null @@ -1,658 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_datatypes.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_datatypes_h__ -#define __include_mpidi_datatypes_h__ - -#ifdef MPIDI_STATISTICS -#include <pami_ext_pe.h> -#endif -#include "mpidi_constants.h" -#include "mpidi_platform.h" -#include "pami.h" -#include "mpidi_trace.h" - -#include "opa_primitives.h" - - - -#if (MPIU_HANDLE_ALLOCATION_METHOD == MPIU_HANDLE_ALLOCATION_THREAD_LOCAL) && defined(__BGQ__) -struct MPIR_Request; -typedef struct -{ - struct MPIR_Request * head; - size_t count; -} MPIDI_RequestHandle_t; -#endif - -#define MPIDI_PT2PT_LIMIT_SET(is_internal,is_immediate,is_local,value) \ - MPIDI_Process.pt2pt.limits_lookup[is_internal][is_immediate][is_local] = value\ - -typedef struct -{ - unsigned remote; - unsigned local; -} MPIDI_remote_and_local_limits_t; - -typedef struct -{ - MPIDI_remote_and_local_limits_t eager; - MPIDI_remote_and_local_limits_t immediate; -} MPIDI_immediate_and_eager_limits_t; - -typedef struct -{ - MPIDI_immediate_and_eager_limits_t application; - MPIDI_immediate_and_eager_limits_t internal; -} MPIDI_pt2pt_limits_t; - -/** - * \brief MPI Process descriptor - * - * This structure contains global configuration flags. - */ -typedef struct -{ - unsigned avail_contexts; - union - { - unsigned limits_array[8]; - unsigned limits_lookup[2][2][2]; - MPIDI_pt2pt_limits_t limits; - } pt2pt; - unsigned disable_internal_eager_scale; /**< The number of tasks at which point eager will be disabled */ -#if TOKEN_FLOW_CONTROL - unsigned long long mp_buf_mem; - unsigned long long mp_buf_mem_max; - unsigned is_token_flow_control_on; -#endif -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - unsigned mp_infolevel; - unsigned mp_statistics; /* print pamid statistcs data */ - unsigned mp_printenv; ; /* print env data */ - unsigned mp_interrupts; ; /* interrupts */ -#endif -#ifdef RDMA_FAILOVER - unsigned mp_s_use_pami_get; /* force the PAMI_Get path instead of PAMI_Rget */ -#endif - -#if (MPIU_HANDLE_ALLOCATION_METHOD == MPIU_HANDLE_ALLOCATION_THREAD_LOCAL) && defined(__BGQ__) - MPIDI_RequestHandle_t request_handles[MPIDI_MAX_THREADS]; -#endif - -#if QUEUE_BINARY_SEARCH_SUPPORT - unsigned queue_binary_search_support_on; -#endif - -#if CUDA_AWARE_SUPPORT - unsigned cuda_aware_support_on; -#endif - - unsigned verbose; /**< The current level of verbosity for end-of-job stats. */ - unsigned statistics; /**< The current level of stats collection. */ - unsigned rma_pending; /**< The max num outstanding requests during an RMA op */ - unsigned shmem_pt2pt; /**< Enable optimized shared memory point-to-point functions. */ - unsigned smp_detect; - pami_geometry_t world_geometry; - - struct - { - unsigned collectives; /**< Enable optimized collective functions. */ - unsigned subcomms; /**< Enable hardware optimized subcomm's */ - unsigned select_colls; /**< Enable collective selection */ - unsigned auto_select_colls; /**< Enable automatic collective selection */ - unsigned memory; /**< Enable memory optimized subcomm's - See MPIR_OPT_LVL_xxxx */ - unsigned num_requests; /**< Number of requests between flow control barriers */ - } - optimized; - - struct - { - volatile unsigned active; /**< Number of contexts with active async progress */ - unsigned mode; /**< 0 == 'disabled', 1 == 'locked', 2 == 'trigger' */ - } - async_progress; - - struct - { - struct - { - unsigned requested; /**< 1 == application requests context post */ - unsigned active; /**< 1 == context post is currently required */ - } context_post; - } perobj; /**< This structure is only used in the 'perobj' mpich lock mode. */ - - unsigned mpir_nbc; /**< Enable MPIR_* non-blocking collectives implementations. */ - int numTasks; /* total number of tasks on a job */ - unsigned typed_onesided; /**< Enable typed PAMI calls for derived types within MPID_Put and MPID_Get. */ -#ifdef DYNAMIC_TASKING - struct MPIDI_PG_t * my_pg; /**< Process group I belong to */ - int my_pg_rank; /**< Rank in process group */ -#endif -} MPIDI_Process_t; - - -enum - { - MPIDI_Protocols_Short, - MPIDI_Protocols_ShortSync, - MPIDI_Protocols_Eager, - MPIDI_Protocols_RVZ, - MPIDI_Protocols_Cancel, - MPIDI_Protocols_Control, - MPIDI_Protocols_WinCtrl, - MPIDI_Protocols_WinAccum, - MPIDI_Protocols_RVZ_zerobyte, - MPIDI_Protocols_WinGetAccum, - MPIDI_Protocols_WinGetAccumAck, -#ifdef DYNAMIC_TASKING - MPIDI_Protocols_Dyntask, - MPIDI_Protocols_Dyntask_disconnect, -#endif - MPIDI_Protocols_WinAtomic, - MPIDI_Protocols_WinAtomicAck, - MPIDI_Protocols_COUNT, - }; - - -/** - * \brief This defines the type of message being sent/received - * mpid_startall() invokes the correct start based on the type of the request - */ -typedef enum - { - MPIDI_REQUEST_PTYPE_RECV, - MPIDI_REQUEST_PTYPE_SEND, - MPIDI_REQUEST_PTYPE_BSEND, - MPIDI_REQUEST_PTYPE_SSEND, - } MPIDI_REQUEST_PTYPE; - - -typedef enum - { - MPIDI_CONTROL_SSEND_ACKNOWLEDGE, - MPIDI_CONTROL_CANCEL_REQUEST, - MPIDI_CONTROL_CANCEL_ACKNOWLEDGE, - MPIDI_CONTROL_CANCEL_NOT_ACKNOWLEDGE, - MPIDI_CONTROL_RENDEZVOUS_ACKNOWLEDGE, - MPIDI_CONTROL_RETURN_TOKENS, - } MPIDI_CONTROL; - - -/** \brief Request completion actions */ -typedef enum - { - MPIDI_CA_COMPLETE, - MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE, /**< Unpack uebuf, then complete. */ - } MPIDI_CA; - - -/** - * \brief MPIDI_Message_match contains enough information to match an - * MPI message. - */ -typedef struct -{ - int tag; /**< match tag */ - int rank; /**< match rank */ - int context_id; /**< match context */ -#ifdef OUT_OF_ORDER_HANDLING - int seqno; /**< match seqno */ -#endif -} MPIDI_Message_match; - - -/** - * \brief MPID pt2pt message header - */ -typedef struct -{ - MPI_Request req; /**< peer's request handle */ - unsigned MPItag; /**< match tag */ - unsigned MPIrank; /**< match rank */ - uint16_t MPIctxt; /**< match context */ - - union { - uint16_t flags; - struct { - unsigned control:3; /**< message type for control protocols */ - unsigned isSync:1; /**< set for sync sends */ - unsigned isRzv :1; /**< use pt2pt rendezvous */ - unsigned noRDMA:1; /**< msg sent via shm or mem reg. fails */ - unsigned reserved:6; /**< unused bits */ - unsigned tokens:4; /** tokens need to be returned */ - } __attribute__ ((__packed__)); - }; - -#ifdef OUT_OF_ORDER_HANDLING - unsigned MPIseqno; /**< match seqno */ -#endif -#if TOKEN_FLOW_CONTROL - unsigned alltokens; /* control:MPIDI_CONTROL_RETURN_TOKENS */ -#endif -} MPIDI_MsgInfo; - -/** \brief Full Rendezvous msg info to be set as two quads of unexpected data. */ -typedef struct -{ - MPIDI_MsgInfo msginfo; - pami_memregion_t memregion; -#ifdef RDMA_FAILOVER - uint32_t memregion_used; -#endif - void * data; - size_t length; -} MPIDI_MsgEnvelope; - -/** \brief This defines the portion of MPIR_Request that is specific to the Device */ -struct MPIDI_Request -{ - struct MPIR_Request *next; /**< Link to next req. in queue */ - struct MPIDU_Datatype*datatype_ptr; /**< Info about the datatype */ - pami_work_t post_request; /**< */ - - MPIDI_MsgEnvelope envelope; - - void *userbuf; /**< User buffer */ - MPI_Aint userbufcount; /**< Userbuf data count */ - MPI_Datatype datatype; /**< Data type of message */ - pami_task_t peer_pami; /**< The other guy's rank (in PAMI) */ - unsigned peer_comm; /**< The other guy's rank (in the orig communicator) */ - unsigned cancel_pending:16; /**< Cancel status */ - unsigned uebuf_malloc:16; /**< does uebuf require free() */ - - unsigned uebuflen; /**< Length (bytes) of uebuf */ - void *uebuf; /**< Unexpected buffer */ - - MPIDI_REQUEST_PTYPE ptype; /**< The persistent msg type */ - MPIDI_CA ca; /**< Completion action */ - pami_memregion_t memregion; /**< Rendezvous recv memregion */ -#ifdef OUT_OF_ORDER_HANDLING - struct MPIR_Request *prev; /**< Link to prev req. in queue */ - void *nextR; /** < pointer to next recv for the out-of-order list, the out-of-order list is a list per source */ - void *prevR; /** < pointer to prev recv for the out-of-order list, the out-of-order list is a list per source */ - struct MPIR_Request *oo_peer; /** < pointer to the matched post recv request to complete in the out-of-order case */ -#endif -#ifdef RDMA_FAILOVER - uint32_t memregion_used:16; - uint32_t shm:16; -#endif -#ifdef MPIDI_TRACE - int partner_id; - int idx; - int PR_idx; -#endif - struct MPIDI_Win_request *win_req; /* anchor of request based rma handle so as to free it properly when wait is called */ -}; - -typedef void* fast_query_t; -/** \brief This defines the portion of MPIR_Comm that is specific to the Device */ -struct MPIDI_Comm -{ - pami_geometry_t geometry; /**< Geometry component for collectives */ - pami_geometry_t parent; /**< The parent geometry this communicator came from */ - pami_algorithm_t *coll_algorithm[PAMI_XFER_COUNT][2]; - pami_metadata_t *coll_metadata[PAMI_XFER_COUNT][2]; - char coll_count[PAMI_XFER_COUNT][2]; - pami_algorithm_t user_selected[PAMI_XFER_COUNT]; - /* no way to tell if user_selected[] is NULL */ - /* could probably union these two though? */ - char user_selected_type[PAMI_XFER_COUNT]; - pami_metadata_t user_metadata[PAMI_XFER_COUNT]; - char last_algorithm[100]; - char preallreduces[MPID_NUM_PREALLREDUCES]; - /* \todo Need to figure out how to deal with algorithms above the pami level */ - char allgathers[4]; - char allgathervs[4]; - char scattervs[2]; - char optgather, optscatter, optreduce; - unsigned num_requests; - /* These need to be freed at geom destroy, so we need to store them - * inside the communicator struct until destroy time rather than - * allocating pointers on the stack - */ - /* For create_taskrange */ - pami_geometry_range_t range; - /* For create_tasklist/endpoints if we ever use it */ - pami_task_t *tasks; - pami_endpoint_t *endpoints; - /* There are some protocols where the optimized protocol always works and - * is the best performance */ - /* Assume we have small vs large cutoffs vs medium for some protocols */ - pami_algorithm_t opt_protocol[PAMI_XFER_COUNT][2]; - int must_query[PAMI_XFER_COUNT][2]; - pami_metadata_t opt_protocol_md[PAMI_XFER_COUNT][2]; - int cutoff_size[PAMI_XFER_COUNT][2]; - /* Our best allreduce protocol always works on - * doubles and sum/min/max. Since that is a common - * occurance let's cache that protocol and call - * it without checking. Any other dt/op must be - * checked */ - pami_algorithm_t cached_allreduce; - pami_metadata_t cached_allreduce_md; - int query_cached_allreduce; - - union tasks_descrip_t { - /* For create_taskrange */ - pami_geometry_range_t *ranges; - /* For create_tasklist/endpoints if we ever use it */ - pami_task_t *tasks; - pami_endpoint_t *endpoints; - } tasks_descriptor; -#ifdef DYNAMIC_TASKING - int local_leader; - long long world_intercomm_cntr; - int *world_ids; /* ids of worlds that composed this communicator (inter communicator created for dynamic tasking */ -#endif - fast_query_t collsel_fast_query; -}; - - -typedef struct -{ - pami_work_t state; - pami_xfer_t *coll_struct; -} MPIDI_Post_coll_t; - - -/** \brief Forward declaration of the MPIR_Comm structure */ -struct MPIR_Comm; -/** \brief Forward declaration of the MPIR_Win structure */ -struct MPIR_Win; -/** \brief Forward declaration of the MPIR_Group structure */ -struct MPIR_Group; - -typedef enum - { - MPIDI_REQUEST_LOCK, - MPIDI_REQUEST_LOCKALL, - } MPIDI_LOCK_TYPE_t; - -struct MPIDI_Win_lock -{ - struct MPIDI_Win_lock *next; - unsigned rank; - MPIDI_LOCK_TYPE_t mtype; /* MPIDI_REQUEST_LOCK or MPIDI_REQUEST_LOCKALL */ - int type; - void *flagAddr; -}; -struct MPIDI_Win_queue -{ - struct MPIDI_Win_lock *head; - struct MPIDI_Win_lock *tail; -}; - -typedef enum { - MPIDI_ACCU_ORDER_RAR = 1, - MPIDI_ACCU_ORDER_RAW = 2, - MPIDI_ACCU_ORDER_WAR = 4, - MPIDI_ACCU_ORDER_WAW = 8 -} MPIDI_Win_info_accumulate_ordering; - -typedef enum { - MPIDI_ACCU_SAME_OP, - MPIDI_ACCU_SAME_OP_NO_OP -} MPIDI_Win_info_accumulate_ops; - -typedef struct MPIDI_Win_info_args { - int no_locks; - MPIDI_Win_info_accumulate_ordering accumulate_ordering; - MPIDI_Win_info_accumulate_ops accumulate_ops; /* default is same_op_no_op */ - int same_size; - int alloc_shared_noncontig; -} MPIDI_Win_info_args; - -typedef struct workQ_t { - void *msgQ; - int count; -} workQ_t; - - -/** - * \brief Collective information related to a window - * - * This structure is used to share information about a local window with - * all nodes in the window communicator. Part of that information includes - * statistics about RMA operations during access/exposure epochs. - * - * The structure is allocated as an array sized for the window communicator. - * Each entry in the array corresponds directly to the node of the same rank. - */ -typedef struct MPIDI_Win_info -{ - void * base_addr; /**< Node's exposure window base address */ - struct MPIR_Win * win; - uint32_t disp_unit; /**< Node's exposure window displacement units */ - pami_memregion_t memregion; /**< Memory region descriptor for each node */ - uint32_t memregion_used; - MPI_Aint base_size; /**< Node's exposure window base size in bytes */ -} MPIDI_Win_info; - -typedef pthread_mutex_t MPIDI_SHM_MUTEX; - -typedef struct MPIDI_Win_shm_ctrl_t -{ - MPIDI_SHM_MUTEX mutex_lock; /* shared memory windows -- lock for */ - /* accumulate/atomic operations */ - OPA_int_t active; - int shm_count; -} MPIDI_Win_shm_ctrl_t; - -typedef struct MPIDI_Win_shm_t -{ - int allocated; /* flag: TRUE iff this window has a shared memory - region associated with it */ - void *base_addr; /* base address of shared memory region */ - MPI_Aint segment_len; /* size of shared memory region */ - union - { - uint32_t shm_id; /* shared memory id - sysv */ - char shm_key[64]; /* shared memory key - posix */ - }; - MPIDI_Win_shm_ctrl_t *ctrl; -} MPIDI_Win_shm_t; - -/** - * \brief Structure of PAMI extensions to MPIR_Win structure - */ -struct MPIDI_Win -{ - struct MPIDI_Win_info *info; /**< allocated array of collective info */ - MPIDI_Win_info_args info_args; - void ** shm_base_addrs; /* base address shared by all process in comm */ - MPIDI_Win_shm_t *shm; /* shared memory info */ - workQ_t work; - int max_ctrlsends; - struct MPIDI_Win_sync - { -#if 0 - /** \todo optimize some of the synchronization assertion */ - uint32_t assert; /**< MPI_MODE_* bits asserted at epoch start */ -#endif - - volatile int origin_epoch_type; /**< curretn epoch type for origin */ - volatile int target_epoch_type; /**< curretn epoch type for target */ - - /* These fields are reset by the sync functions */ - uint32_t total; /**< The number of PAMI requests that we know about (updated only by calling thread) */ - volatile uint32_t started; /**< The number of PAMI requests made (updated only in the context_post callback) */ - volatile uint32_t complete; /**< The number of completed PAMI requests (only updated by the done callbacks) */ - - struct MPIDI_Win_sync_pscw - { - struct MPIR_Group * group; - volatile unsigned count; - } sc, pw; - struct MPIDI_Win_sync_lock - { - struct - { - volatile unsigned locked; - volatile unsigned allLocked; - } remote; - struct - { - struct MPIDI_Win_queue requested; - int type; - unsigned count; - } local; - } lock; - } sync; - int request_based; /* flag for request based rma */ - struct MPIR_Request *rreq; /* anchor of MPIR_Request for request based rma */ -}; - -/** - * \brief Structures and typedefs for collective selection extensions in PAMI - */ - -typedef void* advisor_t; -typedef void* advisor_table_t; -typedef void* advisor_attribute_name_t; - -typedef union -{ - size_t intval; - double doubleval; - const char * chararray; - const size_t * intarray; -} advisor_attribute_value_t; - -typedef struct -{ - advisor_attribute_name_t name; - advisor_attribute_value_t value; -} advisor_configuration_t; - -typedef struct { - pami_xfer_type_t *collectives; - size_t num_collectives; - size_t *procs_per_node; - size_t num_procs_per_node; - size_t *geometry_sizes; - size_t num_geometry_sizes; - size_t *message_sizes; - size_t num_message_sizes; - int iter; - int verify; - int verbose; - int checkpoint; -} advisor_params_t; - -typedef enum -{ - COLLSEL_ALGO = 0, /* 'Always works' PAMI algorithm */ - COLLSEL_QUERY_ALGO, /* 'Must query' PAMI algorithm */ - COLLSEL_EXTERNAL_ALGO, /* External algorithm */ -} advisor_algorithm_type_t; - -/* External algorithm callback function */ -typedef pami_result_t (*external_algorithm_fn)(pami_xfer_t *, void *); - -typedef struct -{ - external_algorithm_fn callback; - void *cookie; -} external_algorithm_t; - -typedef struct -{ - union - { - pami_algorithm_t internal;/* PAMI Algorithm */ - external_algorithm_t external;/* External Algorithm */ - } algorithm; - pami_metadata_t *metadata; - advisor_algorithm_type_t algorithm_type; -} advisor_algorithm_t; - - -typedef pami_result_t (*pami_extension_collsel_init) (pami_client_t, - advisor_configuration_t [], - size_t, - pami_context_t [], - size_t, - advisor_t *); - -typedef pami_result_t (*pami_extension_collsel_destroy) (advisor_t *); - -typedef int (*pami_extension_collsel_initialized) (pami_client_t, advisor_t *); - -typedef pami_result_t (*pami_extension_collsel_table_load) (advisor_t, - char *, - advisor_table_t *); - -typedef pami_result_t (*pami_extension_collsel_get_collectives) (advisor_table_t, - pami_xfer_type_t **, - unsigned *); - -typedef pami_result_t (*pami_extension_collsel_register_algorithms) (advisor_table_t, - pami_geometry_t, - pami_xfer_type_t, - advisor_algorithm_t *, - size_t); - -typedef pami_result_t (*external_geometry_create_fn)(pami_geometry_range_t* task_slices, - size_t slice_count, - pami_geometry_t *geometry, - void **cookie); - -typedef pami_result_t (*external_geometry_destroy_fn)(void *cookie); - -typedef pami_result_t (*external_register_algorithms_fn)(void *cookie, - pami_xfer_type_t collective, - advisor_algorithm_t **algorithms, - size_t *num_algorithms); - -typedef struct -{ - external_geometry_create_fn geometry_create; - external_geometry_destroy_fn geometry_destroy; - external_register_algorithms_fn register_algorithms; -} external_geometry_ops_t; - -typedef pami_result_t (*pami_extension_collsel_table_generate) (advisor_t, - char *, - advisor_params_t *, - external_geometry_ops_t *, - int); - -typedef pami_result_t (*pami_extension_collsel_query_create) (advisor_table_t advisor_table, - pami_geometry_t geometry, - fast_query_t *query); - -typedef pami_result_t (*pami_extension_collsel_query_destroy) (fast_query_t *query); - -typedef int (*pami_extension_collsel_advise) (fast_query_t fast_query, - pami_xfer_type_t xfer_type, - size_t message_size, - advisor_algorithm_t algorithms_optimized[], - size_t max_algorithms); - - - -#endif diff --git a/src/mpid/pamid/include/mpidi_externs.h b/src/mpid/pamid/include/mpidi_externs.h deleted file mode 100644 index 774e5c4..0000000 --- a/src/mpid/pamid/include/mpidi_externs.h +++ /dev/null @@ -1,43 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_externs.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_externs_h__ -#define __include_mpidi_externs_h__ - - -extern pami_client_t MPIDI_Client; -extern pami_context_t MPIDI_Context[]; - -extern MPIDI_Process_t MPIDI_Process; - -extern advisor_table_t MPIDI_Collsel_advisor_table; -extern pami_extension_t MPIDI_Collsel_extension; -extern advisor_params_t MPIDI_Collsel_advisor_params; -extern char *MPIDI_Collsel_output_file; -extern pami_extension_collsel_advise MPIDI_Pamix_collsel_advise; - -#endif diff --git a/src/mpid/pamid/include/mpidi_hooks.h b/src/mpid/pamid/include/mpidi_hooks.h deleted file mode 100644 index 54e47af..0000000 --- a/src/mpid/pamid/include/mpidi_hooks.h +++ /dev/null @@ -1,66 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_hooks.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_hooks_h__ -#define __include_mpidi_hooks_h__ - - -struct MPID_VCR_t { - pami_task_t taskid; -#ifdef DYNAMIC_TASKING - int pg_rank; /** rank in process group **/ - struct MPIDI_PG *pg; /** process group **/ -#endif -}; -typedef struct MPID_VCR_t * MPID_VCR ; -typedef struct MPIDI_VCRT * MPID_VCRT; - - -typedef size_t intptr_t; - -#define MPID_Irsend MPID_Isend -#define MPID_Rsend MPID_Send -#define MPID_Rsend_init MPID_Send_init - - -/** \brief Our progress engine does not require state */ -#define MPID_PROGRESS_STATE_DECL - -/** \brief This defines the portion of MPIR_Request that is specific to the Device */ -#define MPID_DEV_REQUEST_DECL struct MPIDI_Request mpid; - -/** \brief This defines the portion of MPIR_Comm that is specific to the Device */ -#define MPID_DEV_COMM_DECL struct MPIDI_Comm mpid; - -/** \brief This defines the portion of MPIR_Win that is specific to the Device */ -#define MPID_DEV_WIN_DECL struct MPIDI_Win mpid; - -#define MPID_Comm_create_hook(a) ({ int MPIDI_Comm_create (MPIR_Comm *comm); MPIDI_Comm_create (a); }) -#define MPID_Comm_free_hook(a) ({ int MPIDI_Comm_destroy(MPIR_Comm *comm); MPIDI_Comm_destroy(a); }) - - -#endif diff --git a/src/mpid/pamid/include/mpidi_macros.h b/src/mpid/pamid/include/mpidi_macros.h deleted file mode 100644 index 9fd3cb4..0000000 --- a/src/mpid/pamid/include/mpidi_macros.h +++ /dev/null @@ -1,217 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_macros.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_macros_h__ -#define __include_mpidi_macros_h__ - -#include "mpidi_datatypes.h" -#include "mpidi_externs.h" - -#define TOKEN_FLOW_CONTROL_ON (TOKEN_FLOW_CONTROL && MPIU_Token_on()) - -#ifdef TRACE_ON -#ifdef __GNUC__ -#define TRACE_ALL(fd, format, ...) fprintf(fd, "%s:%u (%d) " format, __FILE__, __LINE__, MPIR_Process.comm_world->rank, ##__VA_ARGS__) -#define TRACE_OUT(format, ...) TRACE_ALL(stdout, format, ##__VA_ARGS__) -#define TRACE_ERR(format, ...) TRACE_ALL(stderr, format, ##__VA_ARGS__) -#else -#define TRACE_OUT(format...) fprintf(stdout, format) -#define TRACE_ERR(format...) fprintf(stderr, format) -#endif -#else -#define TRACE_OUT(format...) -#define TRACE_ERR(format...) -#endif - -#if TOKEN_FLOW_CONTROL -#define MPIU_Token_on() (MPIDI_Process.is_token_flow_control_on) -#else -#define MPIU_Token_on() (0) -#endif - -/** - * \brief Gets significant info regarding the datatype - * Used in mpid_send, mpidi_send. - */ -#define MPIDI_Datatype_get_info(_count, _datatype, \ -_dt_contig_out, _data_sz_out, _dt_ptr, _dt_true_lb) \ -({ \ - if (HANDLE_GET_KIND(_datatype) == HANDLE_KIND_BUILTIN) \ - { \ - (_dt_ptr) = NULL; \ - (_dt_contig_out) = TRUE; \ - (_dt_true_lb) = 0; \ - (_data_sz_out) = (_count) * \ - MPIDU_Datatype_get_basic_size(_datatype); \ - } \ - else \ - { \ - MPIDU_Datatype_get_ptr((_datatype), (_dt_ptr)); \ - (_dt_contig_out) = (_dt_ptr)->is_contig; \ - (_dt_true_lb) = (_dt_ptr)->true_lb; \ - (_data_sz_out) = (_count) * (_dt_ptr)->size; \ - } \ -}) - -/** - * \brief Gets data size of the datatype - */ -#define MPIDI_Datatype_get_data_size(_count, _datatype, \ -_data_sz_out) \ -({ \ - if (HANDLE_GET_KIND(_datatype) == HANDLE_KIND_BUILTIN) \ - { \ - (_data_sz_out) = (_count) * \ - MPIDU_Datatype_get_basic_size(_datatype); \ - } \ - else \ - { \ - MPIDU_Datatype*_dt_ptr; \ - MPIDU_Datatype_get_ptr((_datatype), (_dt_ptr)); \ - (_data_sz_out) = (_count) * (_dt_ptr)->size; \ - } \ -}) - -/* Add some error checking for size eventually */ -#define MPIDI_Update_last_algorithm(_comm, _name) \ -({ strncpy( (_comm)->mpid.last_algorithm, (_name), strlen((_name))+1); }) - - -/** - * \brief Macro for allocating memory - * - * \param[in] count Number of elements to allocate - * \param[in] type The type of the memory, excluding "*" - * \return Address or NULL - */ -#define MPL_calloc0(count, type) \ -({ \ - size_t __size = (count) * sizeof(type); \ - type* __p = MPL_malloc(__size); \ - MPID_assert(__p != NULL); \ - if (__p != NULL) \ - memset(__p, 0, __size); \ - __p; \ -}) - -#define MPIU_TestFree(p) \ -({ \ - if (*(p) != NULL) \ - { \ - MPL_free(*(p)); \ - *(p) = NULL; \ - } \ -}) - - -#define MPID_VCR_GET_LPID(vcr, index) \ -({ \ - vcr[index]->taskid; \ -}) - -#define MPID_VCR_GET_LPIDS(comm, taskids) \ -({ \ - int i; \ - taskids=MPL_malloc((comm->local_size)*sizeof(pami_task_t)); \ - MPID_assert(taskids != NULL); \ - for(i=0; i<comm->local_size; i++) \ - taskids[i] = comm->vcr[i]->taskid; \ -}) -#define MPID_VCR_FREE_LPIDS(taskids) MPL_free(taskids) - -#define MPID_GPID_Get(comm_ptr, rank, gpid) \ -({ \ - gpid[1] = MPID_VCR_GET_LPID(comm_ptr->vcr, rank); \ - gpid[0] = 0; \ - MPI_SUCCESS; /* return success from macro */ \ -}) - - -static inline void -MPIDI_Context_post(pami_context_t context, - pami_work_t * work, - pami_work_function fn, - void * cookie) -{ -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - /* It is possible that a work function posted to a context may attempt to - * initiate a communication operation and, if context post were disabled, that - * operation would be performed directly on the context BY TAKING A LOCK that - * the is already held by the thread that is advancing the context. This will - * result in a hang. - * - * A solution would be to identify all code flows where this situation might - * occur and then change the code to avoid taking a lock that is already held. - * - * Another solution is to always disable the "non-context-post" configuration - * when compiled with per-object locking. This would only occur if the - * application requested !MPI_THREAD_MULTIPLE and the "pretend single threaded - * by disabling async progress, context post, and multiple contexts" optimization - * was in effect. - */ - pami_result_t rc; - rc = PAMI_Context_post(context, work, fn, cookie); - MPID_assert(rc == PAMI_SUCCESS); -#else /* (MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ) */ - /* - * It is not necessary to lock the context before access in the "global" - * mpich lock mode because all threads, application and async progress, - * must first acquire the global mpich lock upon entry into the library. - */ - fn(context, cookie); -#endif -} - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) -#define MPIDI_Send_post(__func, __req) \ -({ \ - pami_context_t context = MPIDI_Context_local(__req); \ - \ - if (likely(MPIDI_Process.perobj.context_post.active > 0)) \ - { \ - pami_result_t rc; \ - rc = PAMI_Context_post(context, \ - &(__req)->mpid.post_request, \ - __func, \ - __req); \ - MPID_assert(rc == PAMI_SUCCESS); \ - } \ - else \ - { \ - PAMI_Context_lock(context); \ - __func(context, __req); \ - PAMI_Context_unlock(context); \ - } \ -}) -#else /* (MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ) */ -#define MPIDI_Send_post(__func, __req) \ -({ \ - __func(MPIDI_Context[0], __req); \ -}) -#endif /* #if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) */ - -#endif diff --git a/src/mpid/pamid/include/mpidi_mutex.h b/src/mpid/pamid/include/mpidi_mutex.h deleted file mode 100644 index 0fdb0f4..0000000 --- a/src/mpid/pamid/include/mpidi_mutex.h +++ /dev/null @@ -1,374 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_mutex.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ -#ifndef __include_mpidi_mutex_h__ -#define __include_mpidi_mutex_h__ - -#include <opa_primitives.h> -#include <mpiimpl.h> -#include <malloc.h> - -#define MPIDI_THREAD_ID() Kernel_ProcessorID() - -#if MPIDI_MUTEX_L2_ATOMIC - - - -#define MUTEX_FAIL 0x8000000000000000UL - -#include <spi/include/kernel/location.h> -#include <spi/include/kernel/memory.h> -#include <spi/include/l2/atomic.h> - - -#define MPIDI_MAX_MUTEXES 16 -typedef struct -{ - uint64_t counter; - uint64_t bound; -} MPIDI_Mutex_t; - -extern MPIDI_Mutex_t * MPIDI_Mutex_vector; -extern uint32_t MPIDI_Mutex_counter[MPIDI_MAX_THREADS][MPIDI_MAX_MUTEXES]; -int MPIDI_Mutex_initialize(); - - -/** - * \brief Try to acquire a mutex identified by an index. - * \param[in] m Index of the mutex - * \return 0 Lock successfully acquired - * \return 1 Lock was not acquired - */ -static inline int -MPIDI_Mutex_try_acquire(unsigned m) -{ - -#if MPIDI_MUTEX_RECURSIVE - size_t tid = MPIDI_THREAD_ID(); - MPID_assert(m < MPIDI_MAX_MUTEXES); - if (MPIDI_Mutex_counter[tid][m] >= 1) { - ++MPIDI_Mutex_counter[tid][m]; - return 0; - } -#endif - - MPIDI_Mutex_t *mutex = &(MPIDI_Mutex_vector[m]); - size_t rc = L2_AtomicLoadIncrementBounded(&mutex->counter); - - if (rc == MUTEX_FAIL) - return 1; - -#if MPIDI_MUTEX_RECURSIVE - MPIDI_Mutex_counter[tid][m] = 1; -#endif - return 0; /* Lock succeeded */ -} - - -/** - * \brief Acquire a mutex identified by an index. - * \param[in] m Index of the mutex - * \return 0 Lock successfully acquired - * \return 1 Fail - */ -static inline int -MPIDI_Mutex_acquire(unsigned m) -{ -#if MPIDI_MUTEX_RECURSIVE - size_t tid = MPIDI_THREAD_ID(); - MPID_assert(m < MPIDI_MAX_MUTEXES); - - if (unlikely(MPIDI_Mutex_counter[tid][m] >= 1)) { - ++MPIDI_Mutex_counter[tid][m]; - return 0; - } -#endif - - MPIDI_Mutex_t *mutex = &(MPIDI_Mutex_vector[m]); - size_t rc = 0; - do { - rc = L2_AtomicLoadIncrementBounded(&mutex->counter); - } while (rc == MUTEX_FAIL); - -#if MPIDI_MUTEX_RECURSIVE - MPIDI_Mutex_counter[tid][m] = 1; -#endif - return 0; -} - - -/** - * \brief Release a mutex identified by an index. - * \param[in] m Index of the mutex - * \return 0 Lock successfully released - * \return 1 Fail - */ -static inline int -MPIDI_Mutex_release(unsigned m) -{ -#if MPIDI_MUTEX_RECURSIVE - size_t tid = MPIDI_THREAD_ID(); - MPID_assert(m < MPIDI_MAX_MUTEXES); - /* Verify this thread is the owner of this lock */ - MPID_assert(MPIDI_Mutex_counter[tid][m] > 0); - - --MPIDI_Mutex_counter[tid][m]; - MPID_assert(MPIDI_Mutex_counter[tid][m] >= 0); - if (unlikely(MPIDI_Mutex_counter[tid][m] > 0)) - return 0; /* Future calls will release the lock to other threads */ -#endif - - /* Wait till all the writes in the critical sections from this - thread have completed and invalidates have been delivered */ - //OPA_read_write_barrier(); - - /* Release the lock */ - L2_AtomicStore(&(MPIDI_Mutex_vector[m].counter), 0); - - return 0; -} - - -#define MPIDI_Mutex_sync() OPA_read_write_barrier() - - - -#elif MPIDI_MUTEX_LLSC - - - -#include <spi/include/kernel/location.h> - -#define MPIDI_MAX_MUTEXES 16 -typedef OPA_int_t MPIDI_Mutex_t; -extern MPIDI_Mutex_t MPIDI_Mutex_vector [MPIDI_MAX_MUTEXES]; -extern uint32_t MPIDI_Mutex_counter[MPIDI_MAX_THREADS][MPIDI_MAX_MUTEXES]; - -/** - * \brief Initialize a mutex. - * - * In this API, mutexes are acessed via indices from - * 0..MPIDI_MAX_MUTEXES. The mutexes are recursive - */ -static inline int -MPIDI_Mutex_initialize() -{ - size_t i, j; - for (i=0; i<MPIDI_MAX_MUTEXES; ++i) { - OPA_store_int(&(MPIDI_Mutex_vector[i]), 0); - } - - for (i=0; i<MPIDI_MAX_MUTEXES; ++i) { - for (j=0; j<MPIDI_MAX_THREADS; ++j) { - MPIDI_Mutex_counter[j][i] = 0; - } - } - - return 0; -} - - -/** - * \brief Try to acquire a mutex identified by an index. - * \param[in] m Index of the mutex - * \return 0 Lock successfully acquired - * \return 1 Lock was not acquired - */ -static inline int -MPIDI_Mutex_try_acquire(unsigned m) -{ - register int old_val; - size_t tid = MPIDI_THREAD_ID(); - - MPID_assert(m < MPIDI_MAX_MUTEXES); - - if (MPIDI_Mutex_counter[tid][m] >= 1) { - ++MPIDI_Mutex_counter[tid][m]; - return 0; - } - - MPIDI_Mutex_t *mutex = &(MPIDI_Mutex_vector[m]); - old_val = OPA_LL_int(mutex); - if (old_val != 0) - return 1; /* Lock failed */ - - int rc = OPA_SC_int(mutex, 1); /* returns 0 when SC fails */ - - if (rc == 0) - return 1; /* Lock failed */ - - MPIDI_Mutex_counter[tid][m] = 1; - return 0; /* Lock succeeded */ -} - - -/** - * \brief Acquire a mutex identified by an index. - * \param[in] m Index of the mutex - * \return 0 Lock successfully acquired - * \return 1 Fail - */ -static inline int -MPIDI_Mutex_acquire(unsigned m) -{ - register int old_val; - size_t tid = MPIDI_THREAD_ID(); - - MPID_assert(m < MPIDI_MAX_MUTEXES); - - if (unlikely(MPIDI_Mutex_counter[tid][m] >= 1)) { - ++MPIDI_Mutex_counter[tid][m]; - return 0; - } - - MPIDI_Mutex_t *mutex = &(MPIDI_Mutex_vector[m]); - do { - do { - old_val = OPA_LL_int(mutex); - } while (old_val != 0); - - } while (!OPA_SC_int(mutex, 1)); - - MPIDI_Mutex_counter[tid][m] = 1; - return 0; -} - - -/** - * \brief Release a mutex identified by an index. - * \param[in] m Index of the mutex - * \return 0 Lock successfully released - * \return 1 Fail - */ -static inline int -MPIDI_Mutex_release(unsigned m) -{ - size_t tid = MPIDI_THREAD_ID(); - MPID_assert(m < MPIDI_MAX_MUTEXES); - /* Verify this thread is the owner of this lock */ - MPID_assert(MPIDI_Mutex_counter[tid][m] > 0); - - --MPIDI_Mutex_counter[tid][m]; - MPID_assert(MPIDI_Mutex_counter[tid][m] >= 0); - if (unlikely(MPIDI_Mutex_counter[tid][m] > 0)) - return 0; /* Future calls will release the lock to other threads */ - - /* Wait till all the writes in the critical sections from this - thread have completed and invalidates have been delivered */ - //OPA_read_write_barrier(); - - /* Release the lock */ - OPA_store_int(&(MPIDI_Mutex_vector[m]), 0); - - return 0; -} - - -#define MPIDI_Mutex_sync() OPA_read_write_barrier() - - - -#else - - - -extern pthread_mutex_t MPIDI_Mutex_lock; - -static inline int -MPIDI_Mutex_initialize() -{ - int rc; - - pthread_mutexattr_t attr; - rc = pthread_mutexattr_init(&attr); - MPID_assert(rc == 0); -#if !defined(__AIX__) - extern int pthread_mutexattr_settype(pthread_mutexattr_t *__attr, int __kind) __THROW __nonnull ((1)); -#else - extern int pthread_mutexattr_settype(pthread_mutexattr_t *__attr, int __kind); -#endif -#ifndef __PE__ -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - rc = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP); -#else /*(MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ)*/ - rc = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_ERRORCHECK_NP); -#endif /*(MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ)*/ -#else /* __PE__ */ -#if !defined(__AIX__) - rc = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP); -#else - rc = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE); -#endif -#endif - MPID_assert(rc == 0); - - rc = pthread_mutex_init(&MPIDI_Mutex_lock, &attr); - MPID_assert(rc == 0); - - return 0; -} - - -static inline int -MPIDI_Mutex_try_acquire(unsigned m) -{ - int rc; - rc = pthread_mutex_trylock(&MPIDI_Mutex_lock); - MPID_assert( (rc == 0) || (rc == EBUSY) ); - /* fprintf(stderr, "%s:%u (rc=%d)\n", __FUNCTION__, __LINE__, rc); */ - return rc; -} - - -static inline int -MPIDI_Mutex_acquire(unsigned m) -{ - int rc; - /* fprintf(stderr, "%s:%u\n", __FUNCTION__, __LINE__); */ - rc = pthread_mutex_lock(&MPIDI_Mutex_lock); - /* fprintf(stderr, "%s:%u (rc=%d)\n", __FUNCTION__, __LINE__, rc); */ - MPID_assert(rc == 0); - return rc; -} - - -static inline int -MPIDI_Mutex_release(unsigned m) -{ - int rc; - rc = pthread_mutex_unlock(&MPIDI_Mutex_lock); - /* fprintf(stderr, "%s:%u (rc=%d)\n", __FUNCTION__, __LINE__, rc); */ - MPID_assert(rc == 0); - return rc; -} - - -#define MPIDI_Mutex_sync() - - -#endif - - -#endif diff --git a/src/mpid/pamid/include/mpidi_platform.h b/src/mpid/pamid/include/mpidi_platform.h deleted file mode 100644 index cd95e72..0000000 --- a/src/mpid/pamid/include/mpidi_platform.h +++ /dev/null @@ -1,196 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_platform.h - * \brief ??? - */ - - -#ifndef __include_mpidi_platform_h__ -#define __include_mpidi_platform_h__ - -/* Allow MPICH to detect local tasks */ -#define MPID_USE_NODE_IDS -typedef int32_t MPID_Node_id_t; - -/* Default values */ - -#define MPIDI_MAX_CONTEXTS 64 -/** This is not the real value, but should default to something larger than PAMI_DISPATCH_SEND_IMMEDIATE_MAX */ -#define MPIDI_SHORT_LIMIT 555 -/** This is set to 4 BGQ torus packets (+1, because of the way it is compared) */ -#define MPIDI_EAGER_LIMIT 2049 -/** This is set to 0 which effectively disables the eager protocol for local transfers */ -#define MPIDI_EAGER_LIMIT_LOCAL 0 -/** This is set to 'max unsigned' which effectively never disables internal eager at scale */ -#define MPIDI_DISABLE_INTERNAL_EAGER_SCALE ((unsigned)-1) - -/* Default features */ -#define USE_PAMI_RDMA 1 -#define USE_PAMI_CONSISTENCY PAMI_HINT_ENABLE -#undef OUT_OF_ORDER_HANDLING -#undef DYNAMIC_TASKING -#undef RDMA_FAILOVER -#undef QUEUE_BINARY_SEARCH_SUPPORT - -#define ASYNC_PROGRESS_MODE_DEFAULT 0 - -/* - * The default behavior is to disable (ignore) the 'internal vs application' and - * the 'local vs remote' point-to-point eager limits. - */ -#define MPIDI_PT2PT_EAGER_LIMIT(is_internal,is_local) \ -({ \ - MPIDI_Process.pt2pt.limits_lookup[0][0][0]; \ -}) - -/* - * The default behavior is to disable (ignore) the 'internal vs application' and - * the 'local vs remote' point-to-point short limits. - */ -#define MPIDI_PT2PT_SHORT_LIMIT(is_internal,is_local) \ -({ \ - MPIDI_Process.pt2pt.limits_lookup[0][1][0]; \ -}) - - - -#ifdef __BGQ__ -#undef MPIDI_EAGER_LIMIT_LOCAL -#define MPIDI_EAGER_LIMIT_LOCAL 4097 -#undef MPIDI_EAGER_LIMIT -#define MPIDI_EAGER_LIMIT 4097 -#undef MPIDI_DISABLE_INTERNAL_EAGER_SCALE -#define MPIDI_DISABLE_INTERNAL_EAGER_SCALE (512*1024) -#define MPIDI_MAX_THREADS 64 -#define MPIDI_MUTEX_L2_ATOMIC 1 -#define MPIDI_OPTIMIZED_COLLECTIVE_DEFAULT 1 - -#define PAMIX_IS_LOCAL_TASK -#define PAMIX_IS_LOCAL_TASK_STRIDE (4) -#define PAMIX_IS_LOCAL_TASK_SHIFT (6) -#define MPIDI_SMP_DETECT_DEFAULT 1 -#define TOKEN_FLOW_CONTROL 0 -#define CUDA_AWARE_SUPPORT 0 - -/* - * Enable both the 'internal vs application' and the 'local vs remote' - * point-to-point eager limits. - */ -#undef MPIDI_PT2PT_EAGER_LIMIT -#define MPIDI_PT2PT_EAGER_LIMIT(is_internal,is_local) \ -({ \ - MPIDI_Process.pt2pt.limits_lookup[is_internal][0][is_local]; \ -}) - -/* - * Enable both the 'internal vs application' and the 'local vs remote' - * point-to-point short limits. - */ -#undef MPIDI_PT2PT_SHORT_LIMIT -#define MPIDI_PT2PT_SHORT_LIMIT(is_internal,is_local) \ -({ \ - MPIDI_Process.pt2pt.limits_lookup[is_internal][1][is_local]; \ -}) - - -#undef ASYNC_PROGRESS_MODE_DEFAULT -#define ASYNC_PROGRESS_MODE_DEFAULT 1 - -static const char _ibm_release_version_[] = "V1R2M0"; -#endif - -#ifdef __PE__ - -/* - * This 'maximum contexts' define needs to be changed when mpich on PE - * will support multiple contexts. Currently the PE PAMI allows multiple - * contexts, but the PE mpich code is not set up to use them. - */ -#undef MPIDI_MAX_CONTEXTS -#define MPIDI_MAX_CONTEXTS 1 - -#undef USE_PAMI_CONSISTENCY -#define USE_PAMI_CONSISTENCY PAMI_HINT_DISABLE -#undef MPIDI_SHORT_LIMIT -#define MPIDI_SHORT_LIMIT 256 - sizeof(MPIDI_MsgInfo) -#undef MPIDI_EAGER_LIMIT -#define MPIDI_EAGER_LIMIT 65536 -#undef MPIDI_EAGER_LIMIT_LOCAL -#define MPIDI_EAGER_LIMIT_LOCAL 1048576 -#define OUT_OF_ORDER_HANDLING 1 -#define MPIDI_STATISTICS 1 -#define MPIDI_PRINTENV 1 -#define MPIDI_OPTIMIZED_COLLECTIVE_DEFAULT 0 -#undef USE_PAMI_RDMA -#define RDMA_FAILOVER -#define MPIDI_BANNER 1 -#define MPIDI_NO_ASSERT 1 -#define TOKEN_FLOW_CONTROL 1 -#define DYNAMIC_TASKING 1 -#define CUDA_AWARE_SUPPORT 1 - -/* 'is local task' extension and limits */ -#define PAMIX_IS_LOCAL_TASK -#define PAMIX_IS_LOCAL_TASK_STRIDE (1) -#define PAMIX_IS_LOCAL_TASK_SHIFT (0) -#define MPIDI_SMP_DETECT_DEFAULT 1 -/* - * Enable only the 'local vs remote' point-to-point eager limits. - */ -#undef MPIDI_PT2PT_EAGER_LIMIT -#define MPIDI_PT2PT_EAGER_LIMIT(is_internal,is_local) \ -({ \ - MPIDI_Process.pt2pt.limits_lookup[0][0][is_local]; \ -}) - -/* - * Enable only the 'local vs remote' point-to-point short limits. - */ -#undef MPIDI_PT2PT_SHORT_LIMIT -#define MPIDI_PT2PT_SHORT_LIMIT(is_internal,is_local) \ -({ \ - MPIDI_Process.pt2pt.limits_lookup[0][1][is_local]; \ -}) - - -#undef ASYNC_PROGRESS_MODE_DEFAULT -#define ASYNC_PROGRESS_MODE_DEFAULT 2 - -/* When the Pok build team extracts this file from CMVC, %W% will expand to */ -/* a string with the current release, for example ppe_rbarlx. */ -/* If this file is cloned from GIT then %W% will not be expanded. The */ -/* banner code has accounted for this situation. */ -static const char _ibm_release_version_[] = "%W%"; - -#endif - -#if TOKEN_FLOW_CONTROL -#define BUFFER_MEM_DEFAULT (1<<26) /* 64MB */ -#define BUFFER_MEM_MAX (1<<26) /* 64MB */ -#define ONE_SHARED_SEGMENT (1<<28) /* 256MB */ -#define EAGER_LIMIT_DEFAULT 65536 -#define MAX_BUF_BKT_SIZE (1<<18) /* Max eager_limit is 256K */ -#define MIN_BUF_BKT_SIZE (64) -#define TOKENS_BIT (4) /* 4 bits piggy back to sender */ - /* should be consistent with tokens - defined in MPIDI_MsgInfo */ -#define TOKENS_BITMASK ((1 << TOKENS_BIT)-1) -#endif - -#endif diff --git a/src/mpid/pamid/include/mpidi_prototypes.h b/src/mpid/pamid/include/mpidi_prototypes.h deleted file mode 100644 index fc683ba..0000000 --- a/src/mpid/pamid/include/mpidi_prototypes.h +++ /dev/null @@ -1,460 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_prototypes.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_prototypes_h__ -#define __include_mpidi_prototypes_h__ - -#if CUDA_AWARE_SUPPORT -#include <cuda_runtime_api.h> -#endif - - -/** - * \addtogroup MPID_RECVQ - * \{ - */ -void MPIDI_Recvq_init(); -void MPIDI_Recvq_finalize(); -int MPIDI_Recvq_FU (int s, int t, int c, MPI_Status * status); -MPIR_Request * MPIDI_Recvq_FDUR (MPI_Request req, int source, int tag, int context_id); -int MPIDI_Recvq_FDPR (MPIR_Request * req); -#ifndef OUT_OF_ORDER_HANDLING -MPIR_Request * MPIDI_Recvq_FDP_or_AEU(MPIR_Request *newreq, int s, int t, int c, int * foundp); -MPIR_Request * MPIDI_Recvq_FDU (int source, int tag, int context_id, int * foundp); -MPIR_Request * MPIDI_Recvq_AEU (MPIR_Request *newreq, int source, int tag, int context_id); -#else -MPIR_Request * MPIDI_Recvq_FDP_or_AEU(MPIR_Request *newreq, int s, pami_task_t ps, int t, int c, int sq, int * foundp); -MPIR_Request * MPIDI_Recvq_FDU (int source, pami_task_t pami_source, int tag, int context_id, int * foundp); -MPIR_Request * MPIDI_Recvq_AEU (MPIR_Request *newreq, int source, pami_task_t pami_source, int tag, int context_id, int msg_seqno); -#endif -void MPIDI_Recvq_DumpQueues (int verbose); -#ifdef OUT_OF_ORDER_HANDLING -void MPIDI_Recvq_enqueue_ool (pami_task_t s, MPIR_Request * r); -void MPIDI_Recvq_insert_ool (MPIR_Request *q,MPIR_Request *e); -#endif -/** \} */ - -void MPIDI_Buffer_copy(const void * const sbuf, - MPI_Aint scount, - MPI_Datatype sdt, - int * smpi_errno, - void * const rbuf, - MPI_Aint rcount, - MPI_Datatype rdt, - intptr_t * rsz, - int * rmpi_errno); - -pami_result_t MPIDI_Send_handoff (pami_context_t context, void * sreq); -pami_result_t MPIDI_Ssend_handoff(pami_context_t context, void * sreq); -pami_result_t MPIDI_Isend_handoff(pami_context_t context, void * sreq); -pami_result_t MPIDI_Isend_handoff_internal(pami_context_t context, void * sreq); - -void MPIDI_RecvMsg_procnull(MPIR_Comm * comm, - unsigned is_blocking, - MPI_Status * status, - MPIR_Request ** request); -void MPIDI_RecvMsg_Unexp(MPIR_Request * rreq, void * buf, MPI_Aint count, MPI_Datatype datatype); - -/** - * \defgroup MPID_CALLBACKS MPID callbacks for communication - * - * These calls are used to manage message asynchronous start and completion - * - * \addtogroup MPID_CALLBACKS - * \{ - */ -void MPIDI_SendDoneCB (pami_context_t context, - void * clientdata, - pami_result_t result); - -void MPIDI_RecvShortAsyncCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void MPIDI_RecvShortSyncCB (pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void MPIDI_RecvCB (pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void MPIDI_RecvRzvCB (pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void MPIDI_RecvRzvCB_zerobyte (pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void MPIDI_RecvDoneCB (pami_context_t context, - void * clientdata, - pami_result_t result); -void MPIDI_RecvDoneCB_mutexed(pami_context_t context, - void * clientdata, - pami_result_t result); -void MPIDI_RecvRzvDoneCB (pami_context_t context, - void * cookie, - pami_result_t result); -void MPIDI_RecvRzvDoneCB_zerobyte (pami_context_t context, - void * cookie, - pami_result_t result); -#ifdef DYNAMIC_TASKING -void MPIDI_Recvfrom_remote_world (pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void MPIDI_Recvfrom_remote_world_disconnect (pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -#endif -#ifdef OUT_OF_ORDER_HANDLING -void MPIDI_Recvq_process_out_of_order_msgs(pami_task_t src, pami_context_t context); -int MPIDI_Recvq_search_recv_posting_queue(int src, int tag, int context_id, - MPIR_Request **handleptr ); -#endif - -void MPIDI_Callback_process_unexp(MPIR_Request *newreq, - pami_context_t context, - const MPIDI_MsgInfo * msginfo, - size_t sndlen, - pami_endpoint_t senderendpoint, - const void * sndbuf, - pami_recv_t * recv, - unsigned isSync); -void MPIDI_Callback_process_trunc(pami_context_t context, - MPIR_Request *rreq, - pami_recv_t *recv, - const void *sndbuf); -void MPIDI_Callback_process_userdefined_dt(pami_context_t context, - const void * sndbuf, - size_t sndlen, - MPIR_Request * rreq); -/** \} */ - - -/** \brief Acknowledge an MPI_Ssend() */ -void MPIDI_SyncAck_post(pami_context_t context, MPIR_Request * req, unsigned rank); -pami_result_t MPIDI_SyncAck_handoff(pami_context_t context, void * inputReq); -/** \brief This is the general PT2PT control message call-back */ -void MPIDI_ControlCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void -MPIDI_WinControlCB(pami_context_t context, - void * cookie, - const void * _control, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void -MPIDI_WinAtomicCB(pami_context_t context, - void * cookie, - const void * _control, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void -MPIDI_WinAtomicAckCB(pami_context_t context, - void * cookie, - const void * _control, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); - - -void -MPIDI_WinGetAccumCB(pami_context_t context, - void * cookie, - const void * _control, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -void -MPIDI_WinGetAccumAckCB(pami_context_t context, - void * cookie, - const void * _control, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); - -/** \brief Helper function to complete a rendevous transfer */ -pami_result_t MPIDI_RendezvousTransfer(pami_context_t context, void* rreq); -pami_result_t MPIDI_RendezvousTransfer_SyncAck(pami_context_t context, void* rreq); -pami_result_t MPIDI_RendezvousTransfer_zerobyte(pami_context_t context, void* rreq); - - -int MPIDI_Comm_create (MPIR_Comm *comm); -int MPIDI_Comm_destroy (MPIR_Comm *comm); -void MPIDI_Coll_comm_create (MPIR_Comm *comm); -void MPIDI_Coll_comm_destroy (MPIR_Comm *comm); -void MPIDI_Env_setup (); -void MPIDI_Comm_world_setup (); - -pami_result_t MPIDI_Comm_create_from_pami_geom(pami_geometry_range_t *task_slices, - size_t slice_count, - pami_geometry_t *geometry, - void **cookie); -pami_result_t MPIDI_Comm_destroy_external(void *comm_ext); -pami_result_t MPIDI_Register_algorithms_ext(void *cookie, - pami_xfer_type_t collective, - advisor_algorithm_t **algorithms, - size_t *num_algorithms); -int MPIDI_collsel_pami_tune_parse_params(int argc, char ** argv); -void MPIDI_collsel_pami_tune_cleanup(); -#if CUDA_AWARE_SUPPORT -int CudaMemcpy( void* dst, const void* src, size_t count, int kind ); -int CudaPointerGetAttributes( struct cudaPointerAttributes* attributes, const void* ptr ); -const char * CudaGetErrorString( int error); -#endif -inline bool MPIDI_enable_cuda(); -inline bool MPIDI_cuda_is_device_buf(const void* ptr); -void MPIDI_Coll_Comm_create (MPIR_Comm *comm); -void MPIDI_Coll_Comm_destroy(MPIR_Comm *comm); -void MPIDI_Comm_coll_query (MPIR_Comm *comm); -void MPIDI_Comm_coll_envvars(MPIR_Comm *comm); -void MPIDI_Comm_coll_select(MPIR_Comm *comm); -void MPIDI_Coll_register (void); - -int MPIDO_Bcast(void *buffer, int count, MPI_Datatype dt, int root, MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Bcast_simple(void *buffer, int count, MPI_Datatype dt, int root, MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_bcast(pami_xfer_t *bcast, void *comm); -int MPIDO_Ibcast(void *buffer, int count, MPI_Datatype datatype, int root, MPIR_Comm *comm_ptr, MPIR_Request **request); -int MPIDO_Barrier(MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Barrier_simple(MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_barrier(pami_xfer_t *barrier, void *comm); -int MPIDO_Ibarrier(MPIR_Comm *comm_ptr, MPIR_Request **request); - -int MPIDO_Allreduce(const void *sbuffer, void *rbuffer, int count, - MPI_Datatype datatype, MPI_Op op, MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Allreduce_simple(const void *sbuffer, void *rbuffer, int count, - MPI_Datatype datatype, MPI_Op op, MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_allreduce(pami_xfer_t *allreduce, void *comm); -int MPIDO_Iallreduce(const void *sbuffer, void *rbuffer, int count, - MPI_Datatype datatype, MPI_Op op, MPIR_Comm *comm_ptr, - MPIR_Request ** request); -int MPIDO_Reduce(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, int root, MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Reduce_simple(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, int root, MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_reduce(pami_xfer_t *reduce, void *comm); -int MPIDO_Ireduce(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, int root, MPIR_Comm *comm_ptr, MPIR_Request **request); -int MPIDO_Allgather(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Allgather_simple(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_allgather(pami_xfer_t *allgather, void *comm); -int MPIDO_Iallgather(const void *sendbuf, int sendcount, MPI_Datatype sendtype, void *recvbuf, - int recvcount, MPI_Datatype recvtype, MPIR_Comm *comm_ptr, - MPIR_Request **request); - -int MPIDO_Allgatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *displs, - MPI_Datatype recvtype, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Allgatherv_simple(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *displs, - MPI_Datatype recvtype, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_allgatherv(pami_xfer_t *allgatherv, void *comm); -int MPIDO_Iallgatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *displs, - MPI_Datatype recvtype, MPIR_Comm * comm_ptr, - MPIR_Request ** request); - -int MPIDO_Gather(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Gather_simple(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_gather(pami_xfer_t *gather, void *comm); -int MPIDO_Igather(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, MPIR_Request **request); - -int MPIDO_Gatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *displs, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Gatherv_simple(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *displs, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_gatherv(pami_xfer_t *gatherv, void *comm); -int MPIDO_Igatherv(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *displs, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, MPIR_Request **request); - -int MPIDO_Scan(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Scan_simple(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_scan(pami_xfer_t *scan, void *comm); -int MPIDO_Iscan(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, MPIR_Request **request); - -int MPIDO_Exscan(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Exscan_simple(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Iexscan(const void *sendbuf, void *recvbuf, int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, MPIR_Request **request); - -int MPIDO_Scatter(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Scatter_simple(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_scatter(pami_xfer_t *scatter, void *comm); -int MPIDO_Iscatter(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, MPIR_Request **request); - -int MPIDO_Scatterv(const void *sendbuf, const int *sendcounts, const int *displs, - MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_Scatterv_simple(const void *sendbuf, const int *sendcounts, const int *displs, - MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_scatterv(pami_xfer_t *scatterv, void *comm); -int MPIDO_Iscatterv(const void *sendbuf, const int *sendcounts, const int *displs, - MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - int root, MPIR_Comm * comm_ptr, MPIR_Request **request); - -int MPIDO_Alltoallv(const void *sendbuf, const int *sendcounts, const int *senddispls, - MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *recvdispls, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Alltoallv_simple(const void *sendbuf, const int *sendcounts, const int *senddispls, - MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *recvdispls, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_alltoallv(pami_xfer_t *alltoallv, void *comm); -int MPIDO_Ialltoallv(const void *sendbuf, const int *sendcounts, const int *senddispls, - MPI_Datatype sendtype, - void *recvbuf, const int *recvcounts, const int *recvdispls, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, MPIR_Request **request); - -int MPIDO_Alltoall(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_Alltoall_simple(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, int *mpierrno); -int MPIDO_CSWrapper_alltoall(pami_xfer_t *alltoall, void *comm); -int MPIDO_Ialltoall(const void *sendbuf, int sendcount, MPI_Datatype sendtype, - void *recvbuf, int recvcount, MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, MPIR_Request **request); - -int MPIDO_Ialltoallw(const void *sendbuf, const int *sendcounts, const int *senddispls, - const MPI_Datatype * sendtypes, - void *recvbuf, const int *recvcounts, const int *recvdispls, - const MPI_Datatype * recvtypes, - MPIR_Comm *comm_ptr, MPIR_Request **request); - -int MPIDO_Reduce_scatter(const void *sendbuf, void *recvbuf, int *recvcounts, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm *comm_ptr, int *mpierrno); - -int MPIDO_Reduce_scatter_block(const void *sendbuf, void *recvbuf, int recvcount, - MPI_Datatype datatype, MPI_Op op, MPIR_Comm *comm_ptr, int *mpierrno); - -int MPIDO_Ireduce_scatter_block(const void *sendbuf, void *recvbuf, int recvcount, - MPI_Datatype datatype, MPI_Op op, MPIR_Comm *comm_ptr, - MPIR_Request **request); - -int MPIDO_Ireduce_scatter(const void *sendbuf, void *recvbuf, const int *recvcounts, - MPI_Datatype datatype, MPI_Op op, MPIR_Comm *comm_ptr, MPIR_Request **request); - -int MPIDI_Datatype_to_pami(MPI_Datatype dt, - pami_type_t *pdt, - MPI_Op op, - pami_data_function *pop, - int *musupport); - -int MPIDI_Dtpami_to_dtmpi(pami_type_t pdt, - MPI_Datatype *dt, - pami_data_function pop, - MPI_Op *op); -void MPIDI_Op_to_string(MPI_Op op, char *string); -pami_result_t MPIDI_Pami_post_wrapper(pami_context_t context, void *cookie); - - -void MPIDI_NBC_init (); - - -#endif diff --git a/src/mpid/pamid/include/mpidi_thread.h b/src/mpid/pamid/include/mpidi_thread.h deleted file mode 100644 index f543565..0000000 --- a/src/mpid/pamid/include/mpidi_thread.h +++ /dev/null @@ -1,184 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_thread.h - * \brief ??? - * - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ -#include "mpidi_mutex.h" - - -#ifndef __include_mpidi_thread_h__ -#define __include_mpidi_thread_h__ - - -/** - * ****************************************************************** - * \brief Mutexes for thread/interrupt safety - * ****************************************************************** - */ - -/* This file is included by mpidpre.h, so it is included before mpiimplthread.h. - * This is intentional because it lets us override the critical section macros */ - -#define MPID_DEVICE_DEFINES_THREAD_CS 1 - - -#if (MPICH_THREAD_LEVEL != MPI_THREAD_MULTIPLE) -#error MPICH_THREAD_LEVEL should be MPI_THREAD_MULTIPLE -#endif - -#define MPIU_THREAD_CS_INIT ({ MPIDI_Mutex_initialize(); }) -#define MPIU_THREAD_CS_FINALIZE - -#define MPIU_THREADSAFE_INIT_DECL(_var) static volatile int _var=1 -#define MPIU_THREADSAFE_INIT_BLOCK_BEGIN(_var) \ - MPIU_THREAD_CS_ENTER(INITFLAG,); \ - if (_var) \ - { -#define MPIU_THREADSAFE_INIT_CLEAR(_var) _var=0 -#define MPIU_THREADSAFE_INIT_BLOCK_END(_var) \ - } \ - MPIU_THREAD_CS_EXIT(INITFLAG,) - - -#define MPIU_THREAD_CS_ENTER(name,_context) MPIU_THREAD_CS_##name##_ENTER(_context) -#define MPIU_THREAD_CS_EXIT(name,_context) MPIU_THREAD_CS_##name##_EXIT (_context) -#define MPIU_THREAD_CS_YIELD(name,_context) MPIU_THREAD_CS_##name##_YIELD(_context) -#define MPIU_THREAD_CS_SCHED_YIELD(name,_context) MPIU_THREAD_CS_##name##_SCHED_YIELD(_context) -#define MPIU_THREAD_CS_TRY(name,_context) MPIU_THREAD_CS_##name##_TRY(_context) - -#if MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__GLOBAL - -#define MPIDI_CS_ENTER(m) ({ if (MPIR_ThreadInfo.isThreaded) { MPIDI_Mutex_acquire(m); } }) -#define MPIDI_CS_EXIT(m) ({ if (MPIR_ThreadInfo.isThreaded) { MPIDI_Mutex_sync(); MPIDI_Mutex_release(m); } }) -#define MPIDI_CS_YIELD(m) ({ if (MPIR_ThreadInfo.isThreaded) { MPIDI_Mutex_sync(); MPIDI_Mutex_release(m); MPIDI_Mutex_acquire(m); } }) -#define MPIDI_CS_TRY(m) ({ (0==MPIDI_Mutex_try_acquire(m)); }) -#define MPIDI_CS_SCHED_YIELD(m) ({ if (MPIR_ThreadInfo.isThreaded) { MPIDI_Mutex_sync(); MPIDI_Mutex_release(m); sched_yield(); MPIDI_Mutex_acquire(m); } }) - -/* There is a single, global lock, held for the duration of an MPI call */ -#define MPIU_THREAD_CS_ALLFUNC_ENTER(_context) MPIDI_CS_ENTER(0) -#define MPIU_THREAD_CS_ALLFUNC_EXIT(_context) MPIDI_CS_EXIT (0) -#define MPIU_THREAD_CS_ALLFUNC_YIELD(_context) MPIDI_CS_YIELD(0) -#define MPIU_THREAD_CS_ALLFUNC_SCHED_YIELD(_context) MPIDI_CS_SCHED_YIELD(0) -#define MPIU_THREAD_CS_ALLFUNC_TRY(_context) MPIDI_CS_TRY(0) -#define MPIU_THREAD_CS_INIT_ENTER(_context) MPIDI_Mutex_acquire(0) -#define MPIU_THREAD_CS_INIT_EXIT(_context) MPIDI_Mutex_release(0) - -#define MPIU_THREAD_CS_CONTEXTID_ENTER(_context) -#define MPIU_THREAD_CS_CONTEXTID_EXIT(_context) -#define MPIU_THREAD_CS_CONTEXTID_YIELD(_context) MPIDI_CS_YIELD(0) -#define MPIU_THREAD_CS_CONTEXTID_SCHED_YIELD(_context) MPIDI_CS_SCHED_YIELD(0) -#define MPIU_THREAD_CS_HANDLEALLOC_ENTER(_context) -#define MPIU_THREAD_CS_HANDLEALLOC_EXIT(_context) -#define MPIU_THREAD_CS_HANDLE_ENTER(_context) -#define MPIU_THREAD_CS_HANDLE_EXIT(_context) -#define MPIU_THREAD_CS_INITFLAG_ENTER(_context) -#define MPIU_THREAD_CS_INITFLAG_EXIT(_context) -#define MPIU_THREAD_CS_MEMALLOC_ENTER(_context) -#define MPIU_THREAD_CS_MEMALLOC_EXIT(_context) -#define MPIU_THREAD_CS_MPI_OBJ_ENTER(_context) MPIDI_CS_ENTER(5) -#define MPIU_THREAD_CS_MPI_OBJ_EXIT(_context) MPIDI_CS_EXIT(5) -#define MPIU_THREAD_CS_MSGQUEUE_ENTER(_context) -#define MPIU_THREAD_CS_MSGQUEUE_EXIT(_context) -#define MPIU_THREAD_CS_PAMI_ENTER(_context) -#define MPIU_THREAD_CS_PAMI_EXIT(_context) -#define MPIU_THREAD_CS_ASYNC_ENTER(_context) MPIDI_CS_ENTER(8) -#define MPIU_THREAD_CS_ASYNC_EXIT(_context) MPIDI_CS_EXIT (8) - -#elif MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ - -#define MPIDI_CS_ENTER(m) \ - do { \ - if (likely(MPIR_ThreadInfo.isThreaded)) { \ - MPIDI_Mutex_acquire(m); \ - } \ - } while (0) - -#define MPIDI_CS_EXIT(m) \ - do { \ - if (likely(MPIR_ThreadInfo.isThreaded)) { \ - MPIDI_Mutex_sync(); \ - MPIDI_Mutex_release(m); \ - } \ - } while (0) - -#define MPIDI_CS_YIELD(m) \ - do { \ - if (likely(MPIR_ThreadInfo.isThreaded)) { \ - MPIDI_Mutex_sync(); \ - MPIDI_Mutex_release(m); \ - MPIDI_Mutex_acquire(m); \ - } \ - } while (0) - -#define MPIDI_CS_TRY(m) \ - do { \ - if (likely(MPIR_ThreadInfo.isThreaded)) { \ - MPIDI_Mutex_try_acquire(m); \ - } \ - } while (0) - -#define MPIDI_CS_SCHED_YIELD(m) \ - do { \ - if (likely(MPIR_ThreadInfo.isThreaded)) { \ - MPIDI_Mutex_sync(); \ - MPIDI_Mutex_release(m); \ - sched_yield(); \ - MPIDI_Mutex_acquire(m); \ - } \ - } while (0) - -#define MPIU_THREAD_CS_ALLFUNC_ENTER(_context) -#define MPIU_THREAD_CS_ALLFUNC_EXIT(_context) -#define MPIU_THREAD_CS_ALLFUNC_YIELD(_context) -#define MPIU_THREAD_CS_ALLFUNC_SCHED_YIELD(_context) -#define MPIU_THREAD_CS_ALLFUNC_TRY(_context) (0) - -#define MPIU_THREAD_CS_INIT_ENTER(_context) MPIDI_CS_ENTER(0) -#define MPIU_THREAD_CS_INIT_EXIT(_context) MPIDI_CS_EXIT(0) - -#define MPIU_THREAD_CS_CONTEXTID_ENTER(_context) MPIDI_CS_ENTER(0) -#define MPIU_THREAD_CS_CONTEXTID_EXIT(_context) MPIDI_CS_EXIT (0) -#define MPIU_THREAD_CS_CONTEXTID_YIELD(_context) MPIDI_CS_YIELD(0) -#define MPIU_THREAD_CS_CONTEXTID_SCHED_YIELD(_context) MPIDI_CS_SCHED_YIELD(0) -#define MPIU_THREAD_CS_HANDLEALLOC_ENTER(_context) MPIDI_CS_ENTER(1) -#define MPIU_THREAD_CS_HANDLEALLOC_EXIT(_context) MPIDI_CS_EXIT (1) -#define MPIU_THREAD_CS_HANDLE_ENTER(_context) MPIDI_CS_ENTER(2) -#define MPIU_THREAD_CS_HANDLE_EXIT(_context) MPIDI_CS_EXIT (2) -#define MPIU_THREAD_CS_INITFLAG_ENTER(_context) MPIDI_CS_ENTER(3) -#define MPIU_THREAD_CS_INITFLAG_EXIT(_context) MPIDI_CS_EXIT (3) -#define MPIU_THREAD_CS_MEMALLOC_ENTER(_context) MPIDI_CS_ENTER(4) -#define MPIU_THREAD_CS_MEMALLOC_EXIT(_context) MPIDI_CS_EXIT (4) -#define MPIU_THREAD_CS_MPI_OBJ_ENTER(context_) MPIDI_CS_ENTER(5) -#define MPIU_THREAD_CS_MPI_OBJ_EXIT(context_) MPIDI_CS_EXIT (5) -#define MPIU_THREAD_CS_MSGQUEUE_ENTER(_context) MPIDI_CS_ENTER(6) -#define MPIU_THREAD_CS_MSGQUEUE_EXIT(_context) MPIDI_CS_EXIT (6) -#define MPIU_THREAD_CS_PAMI_ENTER(_context) MPIDI_CS_ENTER(7) -#define MPIU_THREAD_CS_PAMI_EXIT(_context) MPIDI_CS_EXIT (7) -#define MPIU_THREAD_CS_ASYNC_ENTER(_context) MPIDI_CS_ENTER(8) -#define MPIU_THREAD_CS_ASYNC_EXIT(_context) MPIDI_CS_EXIT (8) - -#endif /* MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__GLOBAL */ - - -#endif /* !MPICH_MPIDTHREAD_H_INCLUDED */ diff --git a/src/mpid/pamid/include/mpidi_trace.h b/src/mpid/pamid/include/mpidi_trace.h deleted file mode 100644 index a921e68..0000000 --- a/src/mpid/pamid/include/mpidi_trace.h +++ /dev/null @@ -1,266 +0,0 @@ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_trace.h - * \brief record trace info. for pt2pt comm. - */ -/* - * - * - */ - - -#ifndef __include_mpidi_trace_h__ -#define __include_mpidi_trace_h__ - -#include <sys/time.h> -#include <sys/param.h> - -#ifdef MPIDI_TRACE -#define N_MSGS 1024 -#define SEQMASK N_MSGS-1 -typedef struct { - void *req; /* address of request */ - void *bufadd; /* user's receive buffer address */ - uint msgid; /* msg seqno. */ - unsigned short ctx; /* mpi context id */ - unsigned short dummy; /* reserved */ - uint nMsgs; /* highest msg seqno that arrived in order */ - int tag; - int len; - int rsource; /* source of the message arrived */ - int rtag; /* tag of a received message */ - int rlen; /* len of a received message */ - int rctx; /* context of a received message */ - void * matchedHandle; /* a message with multiple handles */ - union { - uint flags; - struct { -#ifdef __BIG_ENDIAN__ - uint posted:1; /* has the receive posted */ - uint rzv:1; /* rendezvous message ? */ - uint sync:1; /* synchronous message? */ - uint sendAck:1; /* send ack? */ - uint sendFin:1; /* send complete info? */ - uint HH:1; /* header handler */ - uint ool:1; /* the msg arrived out of order */ - uint matchedInOOL:1;/* found a match in out of order list */ - uint comp_in_HH:4; /* the msg completed in header handler */ - uint comp_in_HHV_noMatch:1;/* no matched in header handler EA */ - uint sync_com_in_HH:1; /* sync msg completed in header handler*/ - uint matchedInHH:1; /* found a match in header haldner */ - uint matchedInComp:1;/* found a match in completion handler */ - uint matchedInUQ:2; /* found a match in unexpected queue */ - uint matchedInUQ2:2;/* found a match in unexpected queue */ - uint matchedInWait:1;/* found a match in MPI_Wait() etc. */ - uint ReadySend:1; /* a ready send messsage */ - uint persist:1; /* persist communication */ - uint reserve:1; - uint reserve1:8; -#else - uint reserve1:8; - uint reserve:1; - uint persist:1; /* persist communication */ - uint ReadySend:1; /* a ready send messsage */ - uint matchedInWait:1;/* found a match in MPI_Wait() etc. */ - uint matchedInUQ2:2;/* found a match in unexpected queue */ - uint matchedInUQ:2; /* found a match in unexpected queue */ - uint matchedInComp:1;/* found a match in completion handler */ - uint matchedInHH:1; /* found a match in header haldner */ - uint sync_com_in_HH:1; /* sync msg completed in header handler*/ - uint comp_in_HHV_noMatch:1;/* no matched in header handler EA */ - uint comp_in_HH:4; /* the msg completed in header handler */ - uint matchedInOOL:1;/* found a match in out of order list */ - uint ool:1; /* the msg arrived out of order */ - uint HH:1; /* header handler */ - uint sendFin:1; /* send complete info? */ - uint sendAck:1; /* send ack? */ - uint sync:1; /* synchronous message? */ - uint rzv:1; /* rendezvous message ? */ - uint posted:1; /* has the receive posted */ -#endif - }f; - }fl; -} recv_status; - -typedef struct { - void *req; /* address of request */ - void *bufaddr; /* address of user's send buffer */ - int dest; /* destination of a message */ - int rank; /* rank in a communicator */ - int mode; /* protocol used */ - uint msgid; /* message sequence no. */ - unsigned short sctx; /* context id */ - unsigned short dummy; - int tag; /* tag of a message */ - int len; /* lengh of a message */ - union { - uint flags; - struct { -#ifdef __BIG_ENDIAN__ - uint blocking:1; /* blocking send ? */ - uint sync:1; /* sync message */ - uint sendEnvelop:1; /* envelop send? */ - uint sendShort:1; /* send immediate */ - uint sendEager:1; /* eager send */ - uint sendRzv:1; /* send via renzdvous protocol */ - uint memRegion:1; /* memory is registered */ - uint use_pami_get:1;/* use only PAMI_Get() */ - uint NoComp:4; /* no completion handler */ - uint sendComp:1; /* send complete */ - uint recvAck:1; /* recv an ack from the receiver */ - uint recvFin:1; /* recv complete information */ - uint complSync:1; /* complete sync */ - uint ReadySend:1; /* ready send */ - uint reqXfer:1; /* request message transfer */ - uint persist:1; /* persistent communiation */ - uint reserve:5; - uint reserve1:8; -#else - uint reserve1:8; - uint reserve:5; - uint persist:1; /* persistent communiation */ - uint reqXfer:1; /* request message transfer */ - uint ReadySend:1; /* ready send */ - uint complSync:1; /* complete sync */ - uint recvFin:1; /* recv complete information */ - uint recvAck:1; /* recv an ack from the receiver */ - uint sendComp:1; /* send complete */ - uint NoComp:4; /* no completion handler */ - uint use_pami_get:1;/* use only PAMI_Get() */ - uint memRegion:1; /* memory is registered */ - uint sendRzv:1; /* send via renzdvous protocol */ - uint sendEager:1; /* eager send */ - uint sendShort:1; /* send immediate */ - uint sendEnvelop:1; /* envelop send? */ - uint sync:1; /* sync message */ - uint blocking:1; /* blocking send ? */ -#endif - }f; - }fl; -} send_status; - -typedef struct { - void *req; /* address of a request */ - void *bufadd; /* address of user receive buffer */ - int src_task; /* source PAMI task id */ - int rank; /* rank in a communicator */ - int tag; /* tag of a posted recv */ - int count; /* count of a specified datattype */ - int datatype; - int len; /* length of a receive message */ - uint nMsgs; /* no. of messages have been received */ - uint msgid; /* msg seqno of the matched message */ -#ifdef __BIG_ENDIAN__ - uint sendCtx:16; /* context of incoming msg */ - uint recvCtx:16; /* context of a posted receive */ -#else - uint recvCtx:16; /* context of a posted receive */ - uint sendCtx:16; /* context of incoming msg */ -#endif - union { - uint flags; - struct { -#ifdef __BIG_ENDIAN__ - uint lw:4; /* use lw protocol immediate send */ - uint persist:4; /* persistent communication */ - uint blocking:2; /* blocking receive */ - uint reserve:6; - uint reserve1:16; -#else - uint reserve1:16; - uint reserve:6; - uint blocking:2; /* blocking receive */ - uint persist:4; /* persistent communication */ - uint lw:4; /* use lw protocol immediate send */ -#endif - }f; - }fl; -} posted_recv; - - -typedef struct MPIDI_Trace_buf { - recv_status *R; /* record incoming messages */ - posted_recv *PR; /* record posted receive */ - send_status *S; /* send messages */ - int totPR; /* total no. of poste receive */ -} MPIDI_Trace_buf_t; - -MPIDI_Trace_buf_t *MPIDI_Trace_buf; - - - -#define MPIDI_SET_PR_REC(rreq,buf,ct,dt,pami_id,rank,tag,comm,is_blk) { \ - int idx,src,seqNo,x; \ - if (pami_id != MPI_ANY_SOURCE) \ - src=pami_id; \ - else { \ - src= MPIR_Process.comm_world->rank; \ - } \ - MPIDI_Trace_buf[src].totPR++ ; \ - seqNo=MPIDI_Trace_buf[src].totPR; \ - idx = (seqNo & SEQMASK); \ - memset(&MPIDI_Trace_buf[src].PR[idx],0,sizeof(posted_recv)); \ - MPIDI_Trace_buf[src].PR[idx].src_task= pami_id; \ - MPIDI_Trace_buf[src].PR[idx].rank = rank; \ - MPIDI_Trace_buf[src].PR[idx].bufadd = buf; \ - MPIDI_Trace_buf[src].PR[idx].msgid = seqNo; \ - MPIDI_Trace_buf[src].PR[idx].count = ct; \ - MPIDI_Trace_buf[src].PR[idx].datatype = dt; \ - MPIDI_Trace_buf[src].PR[idx].tag=tag; \ - MPIDI_Trace_buf[src].PR[idx].sendCtx=comm->context_id; \ - MPIDI_Trace_buf[src].PR[idx].recvCtx=comm->recvcontext_id;\ - MPIDI_Trace_buf[src].PR[idx].fl.f.blocking=is_blk; \ - rreq->mpid.PR_idx=idx; \ -} - -#define MPIDI_GET_S_REC(dd,sreq,ctx,isSync,dataSize) { \ - send_status *sstatus; \ - int seqNo=sreq->mpid.envelope.msginfo.MPIseqno; \ - int idx = (seqNo & SEQMASK); \ - sreq->mpid.partner_id=dd; \ - memset(&MPIDI_Trace_buf[dd].S[idx],0,sizeof(send_status));\ - sstatus=&MPIDI_Trace_buf[dd].S[idx]; \ - sstatus->req = (void *)sreq; \ - sstatus->tag = sreq->mpid.envelope.msginfo.MPItag; \ - sstatus->dest = sreq->mpid.peer_pami; \ - sstatus->rank = sreq->mpid.peer_comm; \ - sstatus->msgid = seqNo; \ - sstatus->fl.f.sync = isSync; \ - sstatus->sctx = ctx; \ - sstatus->tag = sreq->mpid.envelope.msginfo.MPItag; \ - sstatus->len= dataSize; \ - sreq->mpid.idx=idx; \ -} - - -#define TRACE_SET_S_BIT(dd,ii,mbr) MPIDI_Trace_buf[(dd)].S[(ii)].mbr=1; -#define TRACE_SET_R_BIT(dd,ii,mbr) MPIDI_Trace_buf[(dd)].R[(ii)].mbr=1; -#define TRACE_SET_S_VAL(dd,ii,mbr,val) MPIDI_Trace_buf[(dd)].S[(ii)].mbr=val; -#define TRACE_SET_R_VALX(dd,rr,mbr,val) { \ - pami_task_t dd1; \ - if (dd < 0) \ - dd1=rr->mpid.partner_id; \ - else \ - dd1=dd; \ - MPIDI_Trace_buf[(dd1)].R[(rr->mpid.PR_idx)].mbr=val; \ -} -#define TRACE_SET_R_VAL(dd,ii,mbr,val) MPIDI_Trace_buf[(dd)].R[(ii)].mbr=val; -#define TRACE_SET_REQ_VAL(ww,val1) ww=val1; -#define TRACE_MEMSET_R(tt,nbr,str) (memset(&MPIDI_Trace_buf[tt].R[(nbr & SEQMASK)],0,sizeof(str))); -#define TRACE_MEMSET_S(tt,nbr,str) (memset(&MPIDI_Trace_buf[tt].S[(nbr & SEQMASK)],0,sizeof(str))); -#else -int recv_status; -int send_status; -int posted_recv; -#define MPIDI_SET_PR_REC(rreq,buf,ct,dt,pami_id,rank,tag,comm,is_blk) -#define MPIDI_GET_S_REC(dest,sreq,ctx,isSync,dataSize) -#define TRACE_SET_S_BIT(dd,ii,mbr) -#define TRACE_SET_R_BIT(dd,ii,mbr) -#define TRACE_SET_S_VAL(dd,ii,mbr,val) -#define TRACE_SET_R_VALX(dd,rr,mbr,val) -#define TRACE_SET_R_VAL(dd,ii,mbr,val) -#define TRACE_SET_REQ_VAL(ww,val1) -#define TRACE_MEMSET_R(tt,nbr,str) - -#endif /* MPIDI_TRACE */ -#endif /* include_mpidi_trace_h */ diff --git a/src/mpid/pamid/include/mpidi_util.h b/src/mpid/pamid/include/mpidi_util.h deleted file mode 100644 index 5a010c4..0000000 --- a/src/mpid/pamid/include/mpidi_util.h +++ /dev/null @@ -1,187 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidi_util.h - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - - -#ifndef __include_mpidi_util_h__ -#define __include_mpidi_util_h__ - - -#if (MPIDI_STATISTICS || MPIDI_PRINTENV || MPIDI_BANNER) -#include <pami_ext_pe.h> -#include <sys/time.h> -#include <sys/param.h> - -#ifndef MAXHOSTNAMELEN -#ifdef LINUX -#define MAXHOSTNAMELEN 64 -#else -#define MAXHOSTNAMELEN 256 -#endif -#endif - -extern int MPIDI_atoi(char* , unsigned int* ); -extern int MPIDI_Banner(char *); -typedef enum {IS_IP, IS_US} MPIDI_TransportType; -typedef enum {NO_STRIPING=0, IS_PACKET_STRIPING} MPIDI_StripingType; - -typedef struct { - char version[128]; /* M19 */ - char hostname[MAXHOSTNAMELEN+1]; -/* Start job info */ - unsigned int partition; - int pid; - int this_task; - int nprocs; - int mp_statistics; - char nnodes[8]; - char tasks_node[8]; - char euilib[8]; - char window_id[128]; - char network_id[32]; - char adapter_type[32]; - char device[8]; - char protocol[16]; - char libpath[MAXPATHLEN]; /* size=BUFSIZE (pm_common.c) */ - char directory[FILENAME_MAX+1]; /* size = 256 */ - int mode_64; - int threaded; - int copySendBufSize:16; - char thread_scope[8]; - char cpu_use[10]; - char adapter_use[10]; - char clock_source[8]; - char priority[24]; - MPIDI_TransportType transport_type; - MPIDI_StripingType strip_type; - int use_rdma; - char *network_string; - int max_pkts_out; - long rdma_min_msg_size; - int timeout; - int interrupts; - uint polling_interval; - unsigned long buffer_mem; - long long buffer_mem_max; - int eager_limit; - int use_token_flow_control; - char wait_mode[8]; - int use_shmem; - uint retransmit_interval; - int shmem_pt2pt; - int shared_mem_pg_size; - char mem_affinity[8]; - int single_thread; - char checkpoint[8]; /* NA */ - char gang_scheduler[10]; /* NA */ - int instances; - char striping_type[40]; - int ack_thresh; - int recv_fifo_sz; - int max_pkt_size; - int rexmit_buf_size; - int rexmit_buf_cnt; - int max_atom_size; - char bulk_min_msg_size[16]; - char use_bulk_xfer[8]; - char user_rdma_avail[16]; - char user_rdma_total[16]; - char debug_notimeout[8]; - char develop[16]; - char stdinmode[12]; - char stdoutmode[12]; - int statistics; - int service_variables; - int rc_max_qp; /* M32 */ - char rc_qp_use_lmc[8]; /* M32 */ - char rc_qp_use_lru[8]; /* M32 */ - char rc_qp_init_setup[8]; /* M32 */ -} MPIDI_printenv_t; - - - -typedef struct { - long sends; /* Count of sends initiated */ - long sendsComplete; /* Count of sends completed (msg sent) */ - long sendWaitsComplete; /* Count of send waits completed - (blocking & nonblocking) */ - long recvs; /* Count of recvs initiated */ - long recvWaitsComplete; /* Count of recv waits complete */ - long earlyArrivals; /* Count of msgs received for which - no receive was posted */ - long earlyArrivalsMatched; /* Count of early arrivals for which - a posted receive has been found */ - long lateArrivals; /* Count of msgs received for which a recv - was posted */ - long unorderedMsgs; /* Total number of out of order msgs */ - long buffer_mem_hwmark; - long pamid_reserve_10; - long pamid_reserve_9; - long pamid_reserve_8; - long pamid_reserve_7; - long pamid_reserve_6; - long pamid_reserve_5; - long pamid_reserve_4; - long pamid_reserve_3; - long pamid_reserve_2; - long pamid_reserve_1; -} MPIX_stats_t; - -extern MPIDI_printenv_t *mpich_env; -extern MPIX_stats_t *mpid_statp; -extern int prtStat; -extern int prtEnv; -extern void set_mpich_env(int *,int*); -extern void MPIDI_open_pe_extension(); -extern void MPIDI_close_pe_extension(); -extern MPIDI_Statistics_write(FILE *); - -#if CUDA_AWARE_SUPPORT -int (*pamidCudaMemcpy)( void* dst, const void* src, size_t count, int kind ); -int (*pamidCudaPointerGetAttributes)( struct cudaPointerAttributes* attributes, const void* ptr ); -const char* (*pamidCudaGetErrorString)( int error ); -extern void * pamidCudaPtr; -#endif -/************************************************************* - * MPIDI_STATISTICS - *************************************************************/ -/* It is not necessary to do fetch_and_and. Statistics */ -/* should be taken when holding a lock. */ -#ifdef MPIDI_STATISTICS - #ifdef _AIX_ - #ifndef __64BIT__ - #define MPID_NSTAT(cmd) if (prtStat) fetch_and_add(&(cmd),1); - #else - #define MPID_NSTAT(cmd) if (prtStat) fetch_and_addlp(&(cmd),1); - #endif - #else /* Linux */ - #define MPID_NSTAT(cmd) if (prtStat) (cmd)++; - #endif /* End Linux */ -#else /* MPIDI_STATISTICS not set*/ - #define MPID_NSTAT(cmd) -#endif /* MPIDI_STATISTICS */ -#endif /* MPIDI_PRINTENV || MPIDI_STATISTICS */ - -#endif /* __include_mpidi_util_h__ */ diff --git a/src/mpid/pamid/include/mpidimpl.h b/src/mpid/pamid/include/mpidimpl.h deleted file mode 100644 index 553b644..0000000 --- a/src/mpid/pamid/include/mpidimpl.h +++ /dev/null @@ -1,197 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidimpl.h - * \brief API MPID additions to MPI functions and structures - */ - -#ifndef __include_mpidimpl_h__ -#define __include_mpidimpl_h__ - -#include <mpiimpl.h> -#include "mpid_sched.h" -#include "pamix.h" -#include <mpix.h> - -extern int MPIDI_PAMID_Timer_is_ready; - -static inline void MPIDI_coll_check_in_place(void* src, void** dst) -{ - if(MPI_IN_PLACE == PAMI_IN_PLACE) - *dst = src; - else - { - if(src == PAMI_IN_PLACE) - *dst = MPI_IN_PLACE; - else - *dst = src; - } -} - -#ifdef DYNAMIC_TASKING - -#define MPIDI_MAX_KVS_VALUE_LEN 4096 - -typedef struct MPIDI_PG -{ - /* MPIU_Object field. MPIDI_PG_t objects are not allocated using the - MPIU_Object system, but we do use the associated reference counting - routines. Therefore, handle must be present, but is not used - except by debugging routines */ - MPIR_OBJECT_HEADER; /* adds handle and ref_count fields */ - - /* Next pointer used to maintain a list of all process groups known to - this process */ - struct MPIDI_PG * next; - - /* Number of processes in the process group */ - int size; - - /* VC table. At present this is a pointer to an array of VC structures. - Someday we may want make this a pointer to an array - of VC references. Thus, it is important to use MPIDI_PG_Get_vc() - instead of directly referencing this field. */ - MPID_VCR vct; - - /* Pointer to the process group ID. The actual ID is defined and - allocated by the process group. The pointer is kept in the - device space because it is necessary for the device to be able to - find a particular process group. */ - void * id; - - /* Replacement abstraction for connection information */ - /* Connection information needed to access processes in this process - group and to share the data with other processes. The items are - connData - pointer for data used to implement these functions - (e.g., a pointer to an array of process group info) - getConnInfo( rank, buf, bufsize, self ) - function to store into - buf the connection information for rank in this process - group - connInfoToString( buf_p, size, self ) - return in buf_p a string - that can be sent to another process to recreate the - connection information (the info needed to support - getConnInfo) - connInfoFromString( buf, self ) - setup the information needed - to implement getConnInfo - freeConnInfo( self ) - free any storage or resources associated - with the connection information. - - See ch3/src/mpidi_pg.c - */ - void *connData; - int (*getConnInfo)( int, char *, int, struct MPIDI_PG * ); - int (*connInfoToString)( char **, int *, struct MPIDI_PG * ); - int (*connInfoFromString)( const char *, struct MPIDI_PG * ); - int (*freeConnInfo)( struct MPIDI_PG * ); -} -MPIDI_PG_t; - -typedef int (*MPIDI_PG_Compare_ids_fn_t)(void * id1, void * id2); -typedef int (*MPIDI_PG_Destroy_fn_t)(MPIDI_PG_t * pg); - - -typedef MPIDI_PG_t * MPIDI_PG_iterator; - -typedef struct conn_info { - int rem_world_id; - int ref_count; - int *rem_taskids; /* The last member of this array is -1 */ - struct conn_info *next; -}conn_info; - -/* link list of transaciton id for all active remote connections in my world */ -typedef struct transactionID_struct { - long long tranid; - int *cntr_for_AM; /* Array size = TOTAL_AM */ - struct transactionID_struct *next; -}transactionID_struct; - -/*-------------------------- - BEGIN MPI PORT SECTION - --------------------------*/ -/* These are the default functions */ -int MPIDI_Comm_connect(const char *, struct MPIR_Info *, int, struct MPIR_Comm *, struct MPIR_Comm **); -int MPIDI_Comm_accept(const char *, struct MPIR_Info *, int, struct MPIR_Comm *, struct MPIR_Comm **); - -int MPIDI_Comm_spawn_multiple(int, char **, char ***, int *, struct MPIR_Info **, - int, struct MPIR_Comm *, struct MPIR_Comm **, int *); - - -typedef struct MPIDI_Port_Ops { - int (*OpenPort)( struct MPIR_Info *, char *); - int (*ClosePort)( const char * ); - int (*CommAccept)( const char *, struct MPIR_Info *, int, struct MPIR_Comm *, - struct MPIR_Comm ** ); - int (*CommConnect)( const char *, struct MPIR_Info *, int, struct MPIR_Comm *, - struct MPIR_Comm ** ); -} MPIDI_PortFns; - - -#define MPIDI_VC_add_ref( _vc ) \ - do { MPIR_Object_add_ref( _vc ); } while (0) - -#define MPIDI_PG_add_ref(pg_) \ -do { \ - MPIR_Object_add_ref(pg_); \ -} while (0) -#define MPIDI_PG_release_ref(pg_, inuse_) \ -do { \ - MPIR_Object_release_ref(pg_, inuse_); \ -} while (0) - -#define MPIDI_VC_release_ref( _vc, _inuse ) \ - do { MPIR_Object_release_ref( _vc, _inuse ); } while (0) - - -/* Initialize a new VC */ -int MPIDI_PG_Create_from_string(const char * str, MPIDI_PG_t ** pg_pptr, - int *flag); -int MPIDI_PG_Get_size(MPIDI_PG_t * pg); -#define MPIDI_PG_Get_size(pg_) ((pg_)->size) -#endif /** DYNAMIC_TASKING **/ - - -static inline pami_endpoint_t MPIDI_Task_to_endpoint(pami_task_t task, size_t offset) -{ - pami_endpoint_t ep; - pami_result_t rc; - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &ep); -#if ASSERT_LEVEL > 0 - if(rc != PAMI_SUCCESS) - MPID_Abort (NULL, 0, 1, "MPIDI_Task_to_endpoint: Invalid task/offset. No endpoint found"); -#endif - return ep; -} - -int -MPIDI_Win_set_info(MPIR_Win *win, - MPIR_Info *info); - -MPI_Aint MPID_Aint_add(MPI_Aint base, MPI_Aint disp); -MPI_Aint MPID_Aint_diff(MPI_Aint addr1, MPI_Aint addr2); - -int MPIDI_Progress_register_hook(int (*progress_fn)(int*), int *id); -int MPIDI_Progress_deregister_hook(int id); -int MPIDI_Progress_activate_hook(int id); -int MPIDI_Progress_deactivate_hook(int id); - -#define MPID_Progress_register_hook(fn_, id_) MPIDI_Progress_register_hook(fn_, id_) -#define MPID_Progress_deregister_hook(id_) MPIDI_Progress_deregister_hook(id_) -#define MPID_Progress_activate_hook(id_) MPIDI_Progress_activate_hook(id_) -#define MPID_Progress_deactivate_hook(id_) MPIDI_Progress_deactivate_hook(id_) -#endif diff --git a/src/mpid/pamid/include/mpidpost.h b/src/mpid/pamid/include/mpidpost.h deleted file mode 100644 index a9ae835..0000000 --- a/src/mpid/pamid/include/mpidpost.h +++ /dev/null @@ -1,104 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidpost.h - * \brief The trailing device header - * - * This file is included after the rest of the headers - * (mpidimpl.h, mpidpre.h, and mpiimpl.h) - */ - -#ifndef __include_mpidpost_h__ -#define __include_mpidpost_h__ - -#include <mpidu_datatype.h> -#include "mpidi_prototypes.h" -#include "mpidi_macros.h" - -#include "../src/mpid_progress.h" -#include "../src/mpid_request.h" -#include "../src/mpid_recvq.h" -#include "../src/pt2pt/mpid_isend.h" -#include "../src/pt2pt/mpid_send.h" -#include "../src/pt2pt/mpid_irecv.h" - -#ifdef DYNAMIC_TASKING -#define MPID_ICCREATE_REMOTECOMM_HOOK(_p,_c,_np,_gp,_r) \ - MPID_PG_ForwardPGInfo(_p,_c,_np,_gp,_r) -#else /* ! DYNAMIC_TASKING */ -/* If DYNAMIC_TASKING is not defined, PAMID does not provide its own GPID routines, - so provide one here. **/ - -/* FIXME: A temporary version for lpids within my comm world */ -static inline int MPID_GPID_GetAllInComm( MPIR_Comm *comm_ptr, int local_size, - int local_gpids[], int *singlePG ) -{ - int i; - int *gpid = local_gpids; - - for (i=0; i<comm_ptr->local_size; i++) { - *gpid++ = 0; - (void)MPID_VCR_Get_lpid( comm_ptr->vcr[i], gpid ); - gpid++; - } - *singlePG = 1; - return 0; -} - -/* FIXME: A temp for lpids within my comm world */ -static inline int MPID_GPID_ToLpidArray( int size, int gpid[], int lpid[] ) -{ - int i; - - for (i=0; i<size; i++) { - lpid[i] = *++gpid; gpid++; - } - return 0; -} -/* FIXME: for MPI1, all process ids are relative to MPI_COMM_WORLD. - For MPI2, we'll need to do something more complex */ -static inline int MPID_VCR_CommFromLpids( MPIR_Comm *newcomm_ptr, - int size, const int lpids[] ) -{ - MPIR_Comm *commworld_ptr; - int i; - - commworld_ptr = MPIR_Process.comm_world; - /* Setup the communicator's vc table: remote group */ - MPID_VCRT_Create( size, &newcomm_ptr->vcrt ); - MPID_VCRT_Get_ptr( newcomm_ptr->vcrt, &newcomm_ptr->vcr ); - for (i=0; i<size; i++) { - /* For rank i in the new communicator, find the corresponding - rank in the comm world (FIXME FOR MPI2) */ - /* printf( "[%d] Remote rank %d has lpid %d\n", - MPIR_Process.comm_world->rank, i, lpids[i] ); */ - if (lpids[i] < commworld_ptr->remote_size) { - MPID_VCR_Dup( commworld_ptr->vcr[lpids[i]], - &newcomm_ptr->vcr[i] ); - } - else { - /* We must find the corresponding vcr for a given lpid */ - /* FIXME: Error */ - return 1; - /* MPID_VCR_Dup( ???, &newcomm_ptr->vcr[i] ); */ - } - } - return 0; -} -#endif /* DYNAMIC_TASKING */ -#endif diff --git a/src/mpid/pamid/include/mpidpre.h b/src/mpid/pamid/include/mpidpre.h deleted file mode 100644 index 6df54a6..0000000 --- a/src/mpid/pamid/include/mpidpre.h +++ /dev/null @@ -1,94 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpidpre.h - * \brief The leading device header - * - * This file is included at the start of the other headers - * (mpidimpl.h, mpidpost.h, and mpiimpl.h). It generally contains - * additions to MPI objects. - */ - -#ifndef __include_mpidpre_h__ -#define __include_mpidpre_h__ - -#include <errno.h> -#include <stdio.h> -#include <assert.h> -#include <pthread.h> -#include <unistd.h> - -#include "mpiimpl.h" -#include "mpid_datatype_fallback.h" -#include <pami.h> - -/* provides "pre" typedefs and such for NBC scheduling mechanism */ -#include "mpidu_pre.h" - -/** \brief Creates a compile error if the condition is false. */ -#define MPID_assert_static(expr) ({ switch(0){case 0:case expr:;} }) -#define MPID_assert_always(x) assert(x) /**< \brief Tests for likely problems--always active */ -#define MPID_abort() assert(0) /**< \brief Always exit--usually implies missing functionality */ -#if ASSERT_LEVEL==0 -#define MPID_assert(x) -#else -#define MPID_assert(x) assert(x) /**< \brief Tests for likely problems--may not be active in performance code */ -#endif - -/* The timer code is allowed to return "NOT_INITIALIZED" before the - * device is initialized. Once the device is initialized, it must - * always return SUCCESS, so the upper layers do not need to check for - * the return code. */ -#define MPID_TIMER_SUCCESS MPL_TIMER_SUCCESS -#define MPID_TIMER_ERR_NOT_INITIALIZED MPL_TIMER_ERR_NOT_INITIALIZED - -typedef double MPID_Time_t; - -int MPID_Wtime(MPID_Time_t *tval); -int MPID_Wtick(double *wtick); -int MPID_Wtime_diff(MPID_Time_t *t1, MPID_Time_t *t2, double *diff); -int MPID_Wtime_todouble(MPID_Time_t *t, double *val); -int MPID_Wtime_acc(MPID_Time_t *t1, MPID_Time_t *t2, MPID_Time_t *t3); -int MPID_Wtime_init(void); - -#include "mpidi_platform.h" - -#include "mpidi_constants.h" -#include "mpidi_datatypes.h" -#include "mpidi_externs.h" -#include "mpidi_hooks.h" -#include "mpidi_thread.h" -#include "mpidi_util.h" - -#ifdef __BGQ__ -#define MPID_HANDLE_NUM_INDICES 256 -#endif /* __BGQ__ */ - -#define MPID_MAX_SMP_BCAST_MSG_SIZE (16384) -#define MPID_MAX_SMP_REDUCE_MSG_SIZE (16384) -#define MPID_MAX_SMP_ALLREDUCE_MSG_SIZE (16384) -#ifdef MPID_DEV_DATATYPE_DECL -#error 'Conflicting definitions of MPID_DEV_DATATYPE_DECL' -#else -#define MPID_DEV_DATATYPE_DECL void *device_datatype; -#endif -#define MPIDU_Dev_datatype_commit_hook(ptr) MPIDI_PAMI_datatype_commit_hook(ptr) -#define MPIDU_Dev_datatype_destroy_hook(ptr) MPIDI_PAMI_datatype_destroy_hook(ptr) -#define MPIDU_Dev_datatype_dup_hook(ptr) MPIDI_PAMI_datatype_dup_hook(ptr) - -#endif diff --git a/src/mpid/pamid/include/mpix.h b/src/mpid/pamid/include/mpix.h deleted file mode 100644 index 6940ad9..0000000 --- a/src/mpid/pamid/include/mpix.h +++ /dev/null @@ -1,298 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/mpix.h - * \brief PAMI device extensions to the MPI Spec - * - * These functions generally use MPI functions and internal APIs to - * expose extra information relating to the specific system on which - * the job is running. This may allow certain hardware specific - * optimizations to be made. - */ - -#ifndef __include_mpix_h__ -#define __include_mpix_h__ - -#if defined(__cplusplus) -extern "C" { -#endif - - - /** - * \brief Print the current system stack - * - * The first frame (this function) is discarded to make the trace look nicer. - */ - void MPIX_Dump_stacks(); - - void MPIX_Progress_poke(); - - /** - * \brief Wait for network to quiesce - * - * \praram[in] timeout Maximum time, Seconds, to wait. 0 for internal default - * \retval MPI_SUCCESS Network appears to be quiesced - * \retval MPI_ERR_PENDING Network did not quiesce - * \retval MPI_ERR_OTHER Encounter error(s), network state unknown - */ - int MPIX_Progress_quiesce(double timeout); - -#define MPIX_TORUS_MAX_DIMS 5 /* This is the maximum physical size of the torus */ - typedef struct - { -/* These fields will be used on all platforms. */ - unsigned prank; /**< Physical rank of the node (irrespective of mapping) */ - unsigned psize; /**< Size of the partition (irrespective of mapping) */ - unsigned ppn; /**< Processes per node */ - unsigned coreID; /**< Process id; values monotonically increase from 0..63 */ - - unsigned clockMHz; /**< Frequency in MegaHertz */ - unsigned memSize; /**< Size of the core memory in MB */ - -/* These fields are only set on torus platforms (i.e. Blue Gene) */ - unsigned torus_dimension; /**< Actual dimension for the torus */ - unsigned Size[MPIX_TORUS_MAX_DIMS]; /**< Max coordinates on the torus */ - unsigned Coords[MPIX_TORUS_MAX_DIMS]; /**< This node's coordinates */ - unsigned isTorus[MPIX_TORUS_MAX_DIMS]; /**< Do we have wraparound links? */ - -/* These fields are only set on systems using Blue Gene IO psets. */ - unsigned rankInPset; - unsigned sizeOfPset; - unsigned idOfPset; - } MPIX_Hardware_t; - - /** - * \brief Determine the rank-in-COMM_WORLD of the process associated with rank-in-comm - * - * Fortran interface: - * MPIX_COMM_RANK2GLOBAL (INTEGER comm, INTEGER crank, INTEGER grank, INTEGER ierr) - * - * \param[in] comm The communicator associated with the input rank - * \param[in] crank The rank-in-comm - * \param[out] grank The rank-in-COMM_WORLD (AKA Global rank) - * \return MPI_SUCCESS on success, an error on failure detection. - */ - int MPIX_Comm_rank2global(MPI_Comm comm, int crank, int *grank); - - /** - * \brief Fill in an MPIX_Hardware_t structure - * \param[in] hw A pointer to an MPIX_Hardware_t structure to be filled in - */ - int MPIX_Hardware(MPIX_Hardware_t *hw); - - - /* These functions only exist on torus platforms (i.e. Blue Gene) */ - - /** - * \brief Determine the number of physical hardware dimensions - * \param[out] numdimensions The number of torus dimensions - * \note This does NOT include the core+thread ID, so if you plan on - * allocating an array based on this information, you'll need to - * add 1 to the value returned here - */ - int MPIX_Torus_ndims(int *numdim); - /** - * \brief Convert an MPI rank into physical coordinates plus core ID - * \param[in] rank The MPI Rank - * \param[out] coords An array of size hw.torus_dimensions+1. The last - * element of the returned array is the core+thread ID - */ - int MPIX_Rank2torus(int rank, int *coords); - /** - * \brief Convert a set of coordinates (physical+core/thread) to an MPI rank - * \param[in] coords An array of size hw.torus_dimensions+1. The last element - * should be the core+thread ID (0..63). - * \param[out] rank The MPI rank cooresponding to the coords array passed in - */ - int MPIX_Torus2rank(int *coords, int *rank); - - /** - * \brief Optimize/deoptimize a communicator by adding/stripping - * platform specific optimizations (i.e. class routes support - * for efficient bcast/reductions). - * \param[in] comm MPI communicator - * \param[in] optimize Optimize(1) or deoptimize(0) the communicator - */ - int MPIX_Comm_update(MPI_Comm comm, int optimize); - - /** - * \brief Return the most recently used collective protocol name - * param[in] comm The communicator that collective was issued on - * param[out] protocol Storage space for the string name - * param[in] length Length available for the string name. - * Note: Max internal length is 100 - */ - int MPIX_Get_last_algorithm_name(MPI_Comm comm, char *protocol, int length); - - /** - * \brief Create a communicator such that all nodes in the same - * communicator are served by the same I/O node - * - * Fortran interface: - * MPIX_PSET_SAME_COMM_CREATE(INTEGER pset_comm, INTEGER ierr) - * - * \note This is a collective operation on MPI_COMM_WORLD - * - * \param [out] pset_comm The new communicator - * - * \return MPI status code - */ - int MPIX_Pset_same_comm_create (MPI_Comm *pset_comm); - - /** - * \brief Create a communicator such that all nodes in the same - * communicator are served by a different I/O node - * - * Fortran interface: - * MPIX_PSET_DIFF_COMM_CREATE(INTEGER pset_comm, INTEGER ierr) - * - * \note This is a collective operation on MPI_COMM_WORLD - * - * \param [out] pset_comm The new communicator - * - * \return MPI status code - */ - int MPIX_Pset_diff_comm_create (MPI_Comm *pset_comm); - - /** - * \brief Create a communicator such that all nodes in the same - * communicator are served by the same I/O node - * - * Fortran interface: - * MPIX_PSET_SAME_COMM_CREATE (INTEGER parent_comm, INTEGER pset_comm, INTEGER ierr) - * - * \note This is a collective operation on the parent communicator. - * - * \param [in] parent_comm The parent communicator - * \param [out] pset_comm The new communicator - * - * \return MPI status code - */ - int MPIX_Pset_same_comm_create_from_parent (MPI_Comm parent_comm, MPI_Comm *pset_comm); - - /** - * \brief Create a communicator such that all nodes in the same - * communicator are served by a different I/O node - * - * Fortran interface: - * MPIX_PSET_DIFF_COMM_CREATE (INTEGER parent_comm, INTEGER pset_comm, INTEGER ierr) - * - * \note This is a collective operation on the parent communicator - * - * \param [in] parent_comm The parent communicator - * \param [out] pset_comm The new communicator - * - * \return MPI status code - */ - int MPIX_Pset_diff_comm_create_from_parent (MPI_Comm parent_comm, MPI_Comm *pset_comm); - - /** - * \brief Retrieve the identifier of the io node associated with the local compute node. - * - * The I/O node identifier is a unique number, yet it is not a monotonically - * increasing integer; such as a rank in a communicator. Multiple ranks, and - * multiple compute nodes, can be associated with the same I/O link. - * - * Fortran interface: - * MPIX_IO_NODE_ID (INTEGER io_node_id) - * - * \note On BG/Q two 'bridge' compute nodes are connected to each io node; - * each unique io node identifier will be associated with at most two - * different io link identifiers. - * - * \return I/O node identifier - */ - int MPIX_IO_node_id (); - - /** - * \brief Retrieve the identifier of the io link associated with the local compute node. - * - * The I/O link identifier is a unique number, yet it is not a monotonically - * increasing integer; such as a rank in a communicator. Multiple ranks, and - * multiple compute nodes, can be associated with the same I/O link. - * - * Fortran interface: - * MPIX_IO_LINK_ID (INTEGER io_link_id) - * - * \return I/O link identifier - */ - int MPIX_IO_link_id (); - - /** - * \brief Retrieve the number of hops to the io node associated with the local compute node. - * - * The distance to the I/O node is the number of hops on the torus from the - * local compute node to the associated I/O node. The 'torus' vs 'mesh' - * attribute of each dimention is considered when determining the distance. - * - * Fortran interface: - * MPIX_IO_DISTANCE (INTEGER io_distance) - * - * \note On BG/Q the 'bridge' compute nodes are those nodes that are closest - * to the I/O node and will have a distance of '1'. - * - * \return number of hops to the I/O node - */ - int MPIX_IO_distance (); - - /** - * \brief Retrieve information about the I/O node associated with the - * local compute node. - * - * \deprecated This function has been replaced with MPIX_IO_link_id() and - * MPIX_IO_distance() - * - * \see MPIX_IO_link_id - * \see MPIX_IO_distance - * \see MPIX_IO_node_id - * - * \param [out] io_node_route_id The unique I/O node route identifier - * \param [out] distance_to_io_node The number of hops to the I/O node - */ - void MPIX_Pset_io_node (int *io_node_route_id, int *distance_to_io_node); - - /** - * \brief Create a Cartesian communicator that exactly matches the partition - * - * This is a collective operation on MPI_COMM_WORLD, and will only run - * successfully on a full partition job (no -np) - * - * The communicator is created to match the size of each dimension, the - * physical coords on each node, and the torus/mesh link status. - * - * Because of MPICH dimension ordering, the associated arrays (i.e. coords, - * sizes, and periods) are in [a, b, c, d, e, t] order. Consequently, when - * using the default ABCDET mapping, the rank in cart_comm will match the rank - * in MPI_COMM_WORLD. However, when using a non-default mapping or a mapfile - * the ranks will be different. - * - * Fortran interface: - * MPIX_CART_COMM_CREATE (INTEGER cart_comm, INTEGER ierr) - * - * \param [out] cart_comm The new Cartesian communicator - * - * \return MPI_SUCCESS or MPI_ERR_TOPOLOGY - */ - int MPIX_Cart_comm_create (MPI_Comm *cart_comm); - - -#if defined(__cplusplus) -} -#endif - -#endif diff --git a/src/mpid/pamid/include/pamix.h b/src/mpid/pamid/include/pamix.h deleted file mode 100644 index 80ef77a..0000000 --- a/src/mpid/pamid/include/pamix.h +++ /dev/null @@ -1,147 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file include/pamix.h - * \brief Extensions to PAMI - */ - - -#ifndef __include_pamix_h__ -#define __include_pamix_h__ - -#include <pami.h> -#include <mpidi_platform.h> -#if defined(__cplusplus) -extern "C" { -#endif - -typedef pami_result_t (*node_info_fn)(pami_task_t task, - uint32_t *node_id, - uint32_t *offset, - uint32_t *max_nodes); -typedef struct -{ - pami_extension_t progress; - - struct - { - pami_extension_t extension; - pami_result_t status; - uint8_t * base; - uintptr_t stride; - uintptr_t bitmask; - node_info_fn node_info; - } is_local_task; - -#if defined(__BGQ__) - pami_extension_t torus; -#endif -} pamix_extension_info_t; - -extern pamix_extension_info_t PAMIX_Extensions; - -void -PAMIX_Initialize(pami_client_t client); - -void -PAMIX_Finalize(pami_client_t client); - -pami_configuration_t -PAMIX_Client_query(pami_client_t client, - pami_attribute_name_t name); - -void -PAMIX_Dispatch_set(pami_context_t context[], - size_t num_contexts, - size_t dispatch, - pami_dispatch_callback_function fn, - pami_dispatch_hint_t options, - size_t * immediate_max); - -pami_task_t -PAMIX_Endpoint_query(pami_endpoint_t endpoint); - - -typedef void (*pamix_progress_function) (pami_context_t context, void *cookie); -#define PAMIX_CLIENT_ASYNC_GUARANTEE 1016 -#define ASYNC_PROGRESS_ALL 0x1111 -typedef enum -{ - PAMIX_PROGRESS_ALL = 0, - PAMIX_PROGRESS_RECV_INTERRUPT = 1, - PAMIX_PROGRESS_TIMER = 2, - PAMIX_PROGRESS_EXT = 1000 -} pamix_progress_t; - -void -PAMIX_Progress_register(pami_context_t context, - pamix_progress_function progress_fn, - pamix_progress_function suspend_fn, - pamix_progress_function resume_fn, - void * cookie); -void -PAMIX_Progress_enable(pami_context_t context, - pamix_progress_t event_type); - -void -PAMIX_Progress_disable(pami_context_t context, - pamix_progress_t event_type); - -pami_result_t -PAMIX_is_local_task_get_node_info(pami_task_t task, - uint32_t *node_id, - uint32_t *offset, - uint32_t *max_nodes); - -#ifdef __BGQ__ - -typedef struct -{ - size_t dims; - size_t *coord; - size_t *size; - size_t *torus; -} pamix_torus_info_t; - -const pamix_torus_info_t * PAMIX_Torus_info(); -int PAMIX_Task2torus(pami_task_t task_id, size_t coords[]); -int PAMIX_Torus2task(size_t coords[], pami_task_t* task_id); - -#endif - -#ifdef PAMIX_IS_LOCAL_TASK -#if defined(PAMIX_IS_LOCAL_TASK_STRIDE) && defined(PAMIX_IS_LOCAL_TASK_SHIFT) -#define PAMIX_Task_is_local(task_id) \ - (((1UL << PAMIX_IS_LOCAL_TASK_SHIFT) & \ - *(PAMIX_Extensions.is_local_task.base + \ - task_id * PAMIX_IS_LOCAL_TASK_STRIDE)) >> PAMIX_IS_LOCAL_TASK_SHIFT) -#else -#define PAMIX_Task_is_local(task_id) \ - ((PAMIX_Extensions.is_local_task.base && \ - (PAMIX_Extensions.is_local_task.bitmask & \ - *(PAMIX_Extensions.is_local_task.base + \ - task_id * PAMIX_Extensions.is_local_task.stride))) > 0) -#endif /* PAMIX_IS_LOCAL_TASK_STRIDE && PAMIX_IS_LOCAL_TASK_SHIFT */ -#else -#define PAMIX_Task_is_local(task_id) (0) -#endif /* PAMIX_IS_LOCAL_TASK */ - -#if defined(__cplusplus) -} -#endif -#endif diff --git a/src/mpid/pamid/src/Makefile.mk b/src/mpid/pamid/src/Makefile.mk deleted file mode 100644 index e2468f3..0000000 --- a/src/mpid/pamid/src/Makefile.mk +++ /dev/null @@ -1,73 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - -#AM_CPPFLAGS += -I$(top_srcdir)/src/mpid/pamid/src/include \ -# -I$(top_builddir)/src/mpid/pamid/src/include - -noinst_HEADERS += \ - src/mpid/pamid/src/mpid_request.h \ - src/mpid/pamid/src/mpid_progress.h \ - src/mpid/pamid/src/mpid_recvq.h - - -include $(top_srcdir)/src/mpid/pamid/src/coll/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/comm/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/misc/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/mpix/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/onesided/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/pamix/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/pt2pt/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/dyntask/Makefile.mk - - -mpi_core_sources += \ - src/mpid/pamid/src/mpid_buffer.c \ - src/mpid/pamid/src/mpidi_bufmm.c \ - src/mpid/pamid/src/mpid_finalize.c \ - src/mpid/pamid/src/mpid_init.c \ - src/mpid/pamid/src/mpid_iprobe.c \ - src/mpid/pamid/src/mpid_probe.c \ - src/mpid/pamid/src/mpid_progress.c \ - src/mpid/pamid/src/mpid_recvq.c \ - src/mpid/pamid/src/mpid_request.c \ - src/mpid/pamid/src/mpid_time.c \ - src/mpid/pamid/src/mpid_vc.c \ - src/mpid/pamid/src/mpidi_env.c \ - src/mpid/pamid/src/mpidi_util.c \ - src/mpid/pamid/src/mpidi_mutex.c \ - src/mpid/pamid/src/mpid_mrecv.c \ - src/mpid/pamid/src/mpid_mprobe.c \ - src/mpid/pamid/src/mpid_imrecv.c \ - src/mpid/pamid/src/mpid_improbe.c \ - src/mpid/pamid/src/mpid_aint.c \ - src/mpid/pamid/src/mpidi_nbc_sched.c \ - src/mpid/pamid/src/mpidi_pami_datatype.c - -if QUEUE_BINARY_SEARCH_SUPPORT -mpi_core_sources += \ - src/mpid/pamid/src/mpid_recvq_mmap.cpp -endif QUEUE_BINARY_SEARCH_SUPPORT - - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/Makefile.mk b/src/mpid/pamid/src/coll/Makefile.mk deleted file mode 100644 index adf7131..0000000 --- a/src/mpid/pamid/src/coll/Makefile.mk +++ /dev/null @@ -1,47 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - -include $(top_srcdir)/src/mpid/pamid/src/coll/barrier/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/bcast/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/allreduce/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/allgather/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/allgatherv/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/scatterv/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/scatter/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/gather/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/alltoall/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/alltoallv/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/gatherv/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/scan/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/reduce/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/alltoallw/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/exscan/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/ired_scat_block/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/ired_scat/Makefile.mk -include $(top_srcdir)/src/mpid/pamid/src/coll/red_scat/Makefile.mk - -mpi_core_sources += \ - src/mpid/pamid/src/coll/coll_utils.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/allgather/Makefile.mk b/src/mpid/pamid/src/coll/allgather/Makefile.mk deleted file mode 100644 index db3c098..0000000 --- a/src/mpid/pamid/src/coll/allgather/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/allgather/mpido_allgather.c \ - src/mpid/pamid/src/coll/allgather/mpido_iallgather.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/allgather/mpido_allgather.c b/src/mpid/pamid/src/coll/allgather/mpido_allgather.c deleted file mode 100644 index 38b46bd..0000000 --- a/src/mpid/pamid/src/coll/allgather/mpido_allgather.c +++ /dev/null @@ -1,820 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/allgather/mpido_allgather.c - * \brief ??? - */ - -/* #define TRACE_ON */ -#include <mpidimpl.h> - - -static void allred_cb_done(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *)clientdata; - (*active)--; -} - -static void allgather_cb_done(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *)clientdata; - (*active)--; -} - - -/* ****************************************************************** */ -/** - * \brief Use (tree) MPIDO_Allreduce() to do a fast Allgather operation - * - * \note This function requires that: - * - The send/recv data types are contiguous - * - Tree allreduce is availible (for max performance) - * - The datatype parameters needed added to the function signature - */ -/* ****************************************************************** */ - -#define MAX_ALLGATHER_ALLREDUCE_BUFFER_SIZE (1024*1024*2) - -int MPIDO_Allgather_allreduce(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPI_Aint send_true_lb, - MPI_Aint recv_true_lb, - size_t send_size, - size_t recv_size, - MPIR_Comm * comm_ptr, - int *mpierrno) - -{ - int rc, i; - char *startbuf = NULL; - char *destbuf = NULL; - const int rank = comm_ptr->rank; - - startbuf = (char *) recvbuf + recv_true_lb; - destbuf = startbuf + rank * send_size; - - if (sendbuf != MPI_IN_PLACE) - { - char *outputbuf = (char *) sendbuf + send_true_lb; - memcpy(destbuf, outputbuf, send_size); - } - - /* TODO: Change to PAMI */ - /*Do a convert and then do the allreudce*/ - if ( recv_size <= MAX_ALLGATHER_ALLREDUCE_BUFFER_SIZE && - (send_size & 0x3)==0 && /*integer/long allgathers only*/ - (sendtype != MPI_DOUBLE || recvtype != MPI_DOUBLE)) - { - double *tmprbuf = (double *)MPL_malloc(recv_size*2); - if (tmprbuf == NULL) - goto direct_algo; /*skip int to fp conversion and go to direct - algo*/ - - double *tmpsbuf = tmprbuf + (rank*send_size)/sizeof(int); - int *sibuf = (int *) destbuf; - - memset(tmprbuf, 0, rank*send_size*2); - memset(tmpsbuf + send_size/sizeof(int), 0, - (recv_size - (rank + 1)*send_size)*2); - - for(i = 0; i < (send_size/sizeof(int)); ++i) - tmpsbuf[i] = (double)sibuf[i]; - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Allreduce(MPI_IN_PLACE, - tmprbuf, - recv_size/sizeof(int), - MPI_DOUBLE, - MPI_SUM, - comm_ptr, - mpierrno); - - sibuf = (int *) startbuf; - for(i = 0; i < (rank*send_size/sizeof(int)); ++i) - sibuf[i] = (int)tmprbuf[i]; - - for(i = (rank+1)*send_size/sizeof(int); i < recv_size/sizeof(int); ++i) - sibuf[i] = (int)tmprbuf[i]; - - MPL_free(tmprbuf); - return rc; - } - - direct_algo: - - memset(startbuf, 0, rank * send_size); - memset(destbuf + send_size, 0, recv_size - (rank + 1) * send_size); - - if (sendtype == MPI_DOUBLE && recvtype == MPI_DOUBLE) - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Allreduce(MPI_IN_PLACE, - startbuf, - recv_size/sizeof(double), - MPI_DOUBLE, - MPI_SUM, - comm_ptr, - mpierrno); - else - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Allreduce(MPI_IN_PLACE, - startbuf, - recv_size/sizeof(int), - MPI_UNSIGNED, - MPI_BOR, - comm_ptr, - mpierrno); - - return rc; -} - - -/* ****************************************************************** */ -/** - * \brief Use (tree/rect) MPIDO_Bcast() to do a fast Allgather operation - * - * \note This function requires one of these (for max performance): - * - Tree broadcast - */ -/* ****************************************************************** */ -int MPIDO_Allgather_bcast(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPI_Aint send_true_lb, - MPI_Aint recv_true_lb, - size_t send_size, - size_t recv_size, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - int i, np, rc = 0; - MPI_Aint extent; - const int rank = comm_ptr->rank; - - np = comm_ptr ->local_size; - MPIDU_Datatype_get_extent_macro(recvtype, extent); - - MPIR_Ensure_Aint_fits_in_pointer ((MPIR_VOID_PTR_CAST_TO_MPI_AINT recvbuf + - np * recvcount * extent)); - if (sendbuf != MPI_IN_PLACE) - { - void *destbuf = recvbuf + rank * recvcount * extent; - MPIR_Localcopy(sendbuf, - sendcount, - sendtype, - destbuf, - recvcount, - recvtype); - } - -/* this code should either abort on first error or somehow aggregate - * error codes, esp since it calls internal routines */ - for (i = 0; i < np; i++) - { - void *destbuf = recvbuf + i * recvcount * extent; - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Bcast(destbuf, - recvcount, - recvtype, - i, - comm_ptr, - mpierrno); - } - - return rc; -} - -/* ****************************************************************** */ -/** - * \brief Use (tree/rect) MPIDO_Alltoall() to do a fast Allgather operation - * - * \note This function requires that: - * - The send/recv data types are contiguous - * - DMA alltoallv is availible (for max performance) - * - The datatype parameters needed added to the function signature - */ -/* ****************************************************************** */ -int MPIDO_Allgather_alltoall(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPI_Aint send_true_lb, - MPI_Aint recv_true_lb, - size_t send_size, - size_t recv_size, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - int i, rc; - void *a2a_sendbuf = NULL; - char *destbuf=NULL; - char *startbuf=NULL; - const int size = comm_ptr->local_size; - const int rank = comm_ptr->rank; - - int a2a_sendcounts[size]; - int a2a_senddispls[size]; - int a2a_recvcounts[size]; - int a2a_recvdispls[size]; - - for (i = 0; i < size; ++i) - { - a2a_sendcounts[i] = send_size; - a2a_senddispls[i] = 0; - a2a_recvcounts[i] = recvcount; - a2a_recvdispls[i] = recvcount * i; - } - if (sendbuf != MPI_IN_PLACE) - { - a2a_sendbuf = (char *)sendbuf + send_true_lb; - } - else - { - startbuf = (char *) recvbuf + recv_true_lb; - destbuf = startbuf + rank * send_size; - a2a_sendbuf = destbuf; - a2a_sendcounts[rank] = 0; - - a2a_recvcounts[rank] = 0; - } - - - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Alltoallv((const void *)a2a_sendbuf, - a2a_sendcounts, - a2a_senddispls, - MPI_CHAR, - recvbuf, - a2a_recvcounts, - a2a_recvdispls, - recvtype, - comm_ptr, - mpierrno); - - return rc; -} - - - - -int -MPIDO_Allgather(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - /* ********************************* - * Check the nature of the buffers - * ********************************* - */ - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - int config[6], i; - MPIDU_Datatype* dt_null = NULL; - MPI_Aint send_true_lb = 0; - MPI_Aint recv_true_lb = 0; - int comm_size = comm_ptr->local_size; - size_t send_bytes = 0; - size_t recv_bytes = 0; - volatile unsigned allred_active = 1; - volatile unsigned allgather_active = 1; - pami_xfer_t allred; - const int rank = comm_ptr->rank; - int queryreq = 0; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const int selected_type = mpid->user_selected_type[PAMI_XFER_ALLGATHER]; - - for (i=0;i<6;i++) config[i] = 1; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - - - allred.cb_done = allred_cb_done; - allred.cookie = (void *)&allred_active; - /* Pick an algorithm that is guaranteed to work for the pre-allreduce */ - /* TODO: This needs selection for fast(er|est) allreduce protocol */ - allred.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLREDUCE][0][0]; - allred.cmd.xfer_allreduce.sndbuf = (void *)config; - allred.cmd.xfer_allreduce.stype = PAMI_TYPE_SIGNED_INT; - allred.cmd.xfer_allreduce.rcvbuf = (void *)config; - allred.cmd.xfer_allreduce.rtype = PAMI_TYPE_SIGNED_INT; - allred.cmd.xfer_allreduce.stypecount = 6; - allred.cmd.xfer_allreduce.rtypecount = 6; - allred.cmd.xfer_allreduce.op = PAMI_DATA_BAND; - - char use_tree_reduce, use_alltoall, use_bcast, use_pami, use_opt; - char *rbuf = NULL, *sbuf = NULL; - - const char * const allgathers = mpid->allgathers; - use_alltoall = allgathers[2]; - use_tree_reduce = allgathers[0]; - use_bcast = allgathers[1]; - use_pami = (selected_type == MPID_COLL_USE_MPICH) ? 0 : 1; - use_opt = use_alltoall || use_tree_reduce || use_bcast || use_pami; - - - TRACE_ERR("flags before: b: %d a: %d t: %d p: %d\n", use_bcast, use_alltoall, use_tree_reduce, use_pami); - if(!use_opt) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH allgather algorithm\n"); - TRACE_ERR("No options set/available; using MPICH for allgather\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHER_MPICH"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(sdt_extent * sendcount); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, sdt_extent * sendcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(rdt_extent * recvcount); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, rdt_extent * recvcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, rdt_extent * recvcount); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Allgather(scbuf, sendcount, sendtype, rcbuf, recvcount, recvtype, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, rdt_extent * recvcount, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Allgather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - } - if ((sendcount < 1 && sendbuf != MPI_IN_PLACE) || recvcount < 1) - return MPI_SUCCESS; - - /* Gather datatype information */ - MPIDI_Datatype_get_info(recvcount, - recvtype, - config[MPID_RECV_CONTIG], - recv_bytes, - dt_null, - recv_true_lb); - - send_bytes = recv_bytes; - rbuf = (char *)recvbuf+recv_true_lb; - - sbuf = PAMI_IN_PLACE; - if(sendbuf != MPI_IN_PLACE) - { - MPIDI_Datatype_get_info(sendcount, - sendtype, - config[MPID_SEND_CONTIG], - send_bytes, - dt_null, - send_true_lb); - sbuf = (char *)sendbuf+send_true_lb; - } - else - if(unlikely(verbose)) - fprintf(stderr,"allgather MPI_IN_PLACE buffering\n"); - - - /* verify everyone's datatype contiguity */ - /* Check buffer alignment now, since we're pre-allreducing anyway */ - /* Only do this if one of the glue protocols is likely to be used */ - if(use_alltoall || use_tree_reduce || use_bcast) - { - config[MPID_ALIGNEDBUFFER] = - !((long)sendbuf & 0x0F) && !((long)recvbuf & 0x0F); - - /* #warning need to determine best allreduce for short messages */ - if(mpid->preallreduces[MPID_ALLGATHER_PREALLREDUCE]) - { - TRACE_ERR("Preallreducing in allgather\n"); - MPIDI_Post_coll_t allred_post; - MPIDI_Context_post(MPIDI_Context[0], &allred_post.state, - MPIDI_Pami_post_wrapper, (void *)&allred); - - MPID_PROGRESS_WAIT_WHILE(allred_active); - } - - - use_alltoall = allgathers[2] && - config[MPID_RECV_CONTIG] && config[MPID_SEND_CONTIG];; - - /* Note: some of the glue protocols use recv_bytes*comm_size rather than - * recv_bytes so we use that for comparison here, plus we pass that in - * to those protocols. */ - use_tree_reduce = allgathers[0] && - config[MPID_RECV_CONTIG] && config[MPID_SEND_CONTIG] && - config[MPID_RECV_CONTINUOUS] && (recv_bytes*comm_size%sizeof(unsigned)) == 0; - - use_bcast = allgathers[1]; - - TRACE_ERR("flags after: b: %d a: %d t: %d p: %d\n", use_bcast, use_alltoall, use_tree_reduce, use_pami); - } - if(use_pami) - { - TRACE_ERR("Using PAMI-level allgather protocol\n"); - pami_xfer_t allgather; - allgather.cb_done = allgather_cb_done; - allgather.cookie = (void *)&allgather_active; - allgather.cmd.xfer_allgather.rcvbuf = rbuf; - allgather.cmd.xfer_allgather.sndbuf = sbuf; - allgather.cmd.xfer_allgather.stype = PAMI_TYPE_BYTE; - allgather.cmd.xfer_allgather.rtype = PAMI_TYPE_BYTE; - allgather.cmd.xfer_allgather.stypecount = send_bytes; - allgather.cmd.xfer_allgather.rtypecount = recv_bytes; - if(selected_type == MPID_COLL_OPTIMIZED) - { - if((mpid->cutoff_size[PAMI_XFER_ALLGATHER][0] == 0) || - (mpid->cutoff_size[PAMI_XFER_ALLGATHER][0] > 0 && mpid->cutoff_size[PAMI_XFER_ALLGATHER][0] >= send_bytes)) - { - allgather.algorithm = mpid->opt_protocol[PAMI_XFER_ALLGATHER][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLGATHER][0]; - queryreq = mpid->must_query[PAMI_XFER_ALLGATHER][0]; - } - else - { - return MPIR_Allgather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - } - } - else - { - allgather.algorithm = mpid->user_selected[PAMI_XFER_ALLGATHER]; - my_md = &mpid->user_metadata[PAMI_XFER_ALLGATHER]; - queryreq = selected_type; - } - - if(unlikely( queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("Querying allgather protocol %s, type was: %d\n", - my_md->name, - selected_type); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; - if(my_md->check_correct.values.rangeminmax) - { - if((my_md->range_lo <= recv_bytes) && - (my_md->range_hi >= recv_bytes)) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%zu) outside range (%zu<->%zu) for %s.\n", - recv_bytes, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&allgather); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH allgather\n", - my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHER_MPICH"); - return MPIR_Allgather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for allgather on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - TRACE_ERR("Calling PAMI_Collective with allgather structure\n"); - MPIDI_Post_coll_t allgather_post; - MPIDI_Context_post(MPIDI_Context[0], &allgather_post.state, MPIDI_Pami_post_wrapper, (void *)&allgather); - - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - MPID_PROGRESS_WAIT_WHILE(allgather_active); - TRACE_ERR("Allgather done\n"); - return PAMI_SUCCESS; - } - - if(use_tree_reduce) - { - if(unlikely(verbose)) - fprintf(stderr,"Using protocol GLUE_ALLREDUCE for allgather\n"); - TRACE_ERR("Using allgather via allreduce\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHER_OPT_ALLREDUCE"); - return MPIDO_Allgather_allreduce(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - send_true_lb, recv_true_lb, send_bytes, recv_bytes*comm_size, comm_ptr, mpierrno); - } - if(use_alltoall) - { - if(unlikely(verbose)) - fprintf(stderr,"Using protocol GLUE_BCAST for allgather\n"); - TRACE_ERR("Using allgather via alltoall\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHER_OPT_ALLTOALL"); - return MPIDO_Allgather_alltoall(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - send_true_lb, recv_true_lb, send_bytes, recv_bytes*comm_size, comm_ptr, mpierrno); - } - - if(use_bcast) - { - if(unlikely(verbose)) - fprintf(stderr,"Using protocol GLUE_ALLTOALL for allgather\n"); - TRACE_ERR("Using allgather via bcast\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHER_OPT_BCAST"); - return MPIDO_Allgather_bcast(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - send_true_lb, recv_true_lb, send_bytes, recv_bytes*comm_size, comm_ptr, mpierrno); - } - - /* Nothing used yet; dump to MPICH */ - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH allgather algorithm\n"); - TRACE_ERR("Using allgather via mpich\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHER_MPICH"); - return MPIR_Allgather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); -} - - -int -MPIDO_Allgather_simple(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - /* ********************************* - * Check the nature of the buffers - * ********************************* - */ - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - MPIDU_Datatype* dt_null = NULL; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - MPI_Aint send_true_lb = 0; - MPI_Aint recv_true_lb = 0; - int snd_data_contig = 1, rcv_data_contig = 1; - size_t send_size = 0; - size_t recv_size = 0; - MPIDU_Segment segment; - volatile unsigned allgather_active = 1; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - const pami_metadata_t *my_md; - - char *rbuf = NULL, *sbuf = NULL; - - - if ((sendcount < 1 && sendbuf != MPI_IN_PLACE) || recvcount < 1) - return MPI_SUCCESS; - - /* Gather datatype information */ - MPIDI_Datatype_get_info(recvcount, - recvtype, - rcv_data_contig, - recv_size, - dt_null, - recv_true_lb); - - send_size = recv_size; - - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_ALLGATHER, send_size, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Allgather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - rbuf = (char *)recvbuf+recv_true_lb; - - if(!rcv_data_contig) - { - rcv_noncontig_buff = MPL_malloc(recv_size * size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendbuf == MPI_IN_PLACE) - { - sbuf = PAMI_IN_PLACE; - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - MPIR_Localcopy(recvbuf + (rank*recvcount*extent), recvcount, recvtype, - rcv_noncontig_buff + (rank*recv_size), recv_size,MPI_CHAR); - } - } - - if(sendbuf != MPI_IN_PLACE) - { - MPIDI_Datatype_get_info(sendcount, - sendtype, - snd_data_contig, - send_size, - dt_null, - send_true_lb); - - sbuf = (char *)sendbuf+send_true_lb; - - if(!snd_data_contig) - { - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - DLOOP_Offset last = send_size; - MPIDU_Segment_init(sendbuf, sendcount, sendtype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, snd_noncontig_buff); - } - } - else - sbuf = PAMI_IN_PLACE; - - TRACE_ERR("Using PAMI-level allgather protocol\n"); - pami_xfer_t allgather; - allgather.cb_done = allgather_cb_done; - allgather.cookie = (void *)&allgather_active; - allgather.cmd.xfer_allgather.rcvbuf = rbuf; - allgather.cmd.xfer_allgather.sndbuf = sbuf; - allgather.cmd.xfer_allgather.stype = PAMI_TYPE_BYTE;/* stype is ignored when sndbuf == PAMI_IN_PLACE */ - allgather.cmd.xfer_allgather.rtype = PAMI_TYPE_BYTE; - allgather.cmd.xfer_allgather.stypecount = send_size; - allgather.cmd.xfer_allgather.rtypecount = recv_size; - allgather.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLGATHER][0][0]; - my_md = &mpid->coll_metadata[PAMI_XFER_ALLGATHER][0][0]; - - TRACE_ERR("Calling PAMI_Collective with allgather structure\n"); - MPIDI_Post_coll_t allgather_post; - MPIDI_Context_post(MPIDI_Context[0], &allgather_post.state, MPIDI_Pami_post_wrapper, (void *)&allgather); - TRACE_ERR("Allgather %s\n", MPIDI_Process.context_post.active>0?"posted":"invoked"); - - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - MPID_PROGRESS_WAIT_WHILE(allgather_active); - if(!rcv_data_contig) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size * size, MPI_CHAR, - recvbuf, recvcount, recvtype); - MPL_free(rcv_noncontig_buff); - } - if(!snd_data_contig) MPL_free(snd_noncontig_buff); - TRACE_ERR("Allgather done\n"); - return MPI_SUCCESS; -} - - -int -MPIDO_CSWrapper_allgather(pami_xfer_t *allgather, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *sbuf; - MPIDI_coll_check_in_place(allgather->cmd.xfer_allgather.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( allgather->cmd.xfer_allgather.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( allgather->cmd.xfer_allgather.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Allgather(sbuf, - allgather->cmd.xfer_allgather.stypecount, sendtype, - allgather->cmd.xfer_allgather.rcvbuf, - allgather->cmd.xfer_allgather.rtypecount, recvtype, - comm_ptr, &mpierrno); - if(allgather->cb_done && rc == 0) - allgather->cb_done(NULL, allgather->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/allgather/mpido_iallgather.c b/src/mpid/pamid/src/coll/allgather/mpido_iallgather.c deleted file mode 100644 index 1f3f3ab..0000000 --- a/src/mpid/pamid/src/coll/allgather/mpido_iallgather.c +++ /dev/null @@ -1,73 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/allgather/mpido_iallgather.c - * \brief ??? - */ - -/*#define TRACE_ON */ -#include <mpidimpl.h> - -int -MPIDO_Iallgather(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPIR_Comm * comm_ptr, - MPIR_Request ** request) -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iallgather_impl() function to invoke the mpich nbc implementation - * of MPI_Iallgather(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iallgather() with a blocking MPIR_Allgather(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH iallgather algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Allgather_impl(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, &mpierrno); - - /* - * The blocking allgather has completed - create and complete a - * MPIR_Request object so the MPIR_Iallgather_impl() function does not - * perform an additional iallgather. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/allgatherv/Makefile.mk b/src/mpid/pamid/src/coll/allgatherv/Makefile.mk deleted file mode 100644 index c8895ca..0000000 --- a/src/mpid/pamid/src/coll/allgatherv/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/allgatherv/mpido_allgatherv.c \ - src/mpid/pamid/src/coll/allgatherv/mpido_iallgatherv.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/allgatherv/mpido_allgatherv.c b/src/mpid/pamid/src/coll/allgatherv/mpido_allgatherv.c deleted file mode 100644 index 1df9876..0000000 --- a/src/mpid/pamid/src/coll/allgatherv/mpido_allgatherv.c +++ /dev/null @@ -1,921 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/allgatherv/mpido_allgatherv.c - * \brief ??? - */ - -/* #define TRACE_ON */ -#include <mpidimpl.h> - -static void allgatherv_cb_done(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *)clientdata; - (*active)--; -} - -static void allred_cb_done(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *)clientdata; - (*active)--; -} - - -/* ****************************************************************** */ -/** - * \brief Use (tree) MPIDO_Allreduce() to do a fast Allgatherv operation - * - * \note This function requires that: - * - The send/recv data types are contiguous - * - The recv buffer is continuous - * - Tree allreduce is availible (for max performance) - */ -/* ****************************************************************** */ -#define MAX_ALLGATHERV_ALLREDUCE_BUFFER_SIZE (1024*1024*2) -int MPIDO_Allgatherv_allreduce(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - int buffer_sum, - const int *displs, - MPI_Datatype recvtype, - MPI_Aint send_true_lb, - MPI_Aint recv_true_lb, - size_t send_size, - size_t recv_size, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - int start, rc, i; - int length; - char *startbuf = NULL; - char *destbuf = NULL; - const int rank = comm_ptr->rank; - TRACE_ERR("Entering MPIDO_Allgatherv_allreduce\n"); - - startbuf = (char *) recvbuf + recv_true_lb; - destbuf = startbuf + displs[rank] * recv_size; - - if (sendbuf != MPI_IN_PLACE) - { - char *outputbuf = (char *) sendbuf + send_true_lb; - memcpy(destbuf, outputbuf, send_size); - } - - //printf("buffer_sum %d, send_size %d recv_size %d\n", buffer_sum, - // (int)send_size, (int)recv_size); - - /* TODO: Change to PAMI */ - /*integer/long/double allgathers only*/ - /*Do a convert and then do the allreudce*/ - if ( buffer_sum <= MAX_ALLGATHERV_ALLREDUCE_BUFFER_SIZE && - (send_size & 0x3)==0 && (recv_size & 0x3)==0) - { - double *tmprbuf = (double *)MPL_malloc(buffer_sum*2); - if (tmprbuf == NULL) - goto direct_algo; /*skip int to fp conversion and go to direct - algo*/ - - double *tmpsbuf = tmprbuf + (displs[rank]*recv_size)/sizeof(int); - int *sibuf = (int *) destbuf; - - memset(tmprbuf, 0, displs[rank]*recv_size*2); - start = (displs[rank] + recvcounts[rank]) * recv_size; - length = buffer_sum - (displs[rank] + recvcounts[rank]) * recv_size; - memset(tmprbuf + start/sizeof(int), 0, length*2); - - for(i = 0; i < (send_size/sizeof(int)); ++i) - tmpsbuf[i] = (double)sibuf[i]; - - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Allreduce(MPI_IN_PLACE, - tmprbuf, - buffer_sum/sizeof(int), - MPI_DOUBLE, - MPI_SUM, - comm_ptr, - mpierrno); - - sibuf = (int *) startbuf; - for(i = 0; i < (displs[rank]*recv_size/sizeof(int)); ++i) - sibuf[i] = (int)tmprbuf[i]; - - for(i = start/sizeof(int); i < buffer_sum/sizeof(int); ++i) - sibuf[i] = (int)tmprbuf[i]; - - MPL_free(tmprbuf); - return rc; - } - - direct_algo: - - start = 0; - length = displs[rank] * recv_size; - memset(startbuf + start, 0, length); - - start = (displs[rank] + - recvcounts[rank]) * recv_size; - length = buffer_sum - (displs[rank] + - recvcounts[rank]) * recv_size; - memset(startbuf + start, 0, length); - - TRACE_ERR("Calling MPIDO_Allreduce from MPIDO_Allgatherv_allreduce\n"); - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Allreduce(MPI_IN_PLACE, - startbuf, - buffer_sum/sizeof(unsigned), - MPI_UNSIGNED, - MPI_BOR, - comm_ptr, - mpierrno); - - TRACE_ERR("Leaving MPIDO_Allgatherv_allreduce\n"); - return rc; -} - -/* ****************************************************************** */ -/** - * \brief Use (tree/rect) MPIDO_Bcast() to do a fast Allgatherv operation - * - * \note This function requires one of these (for max performance): - * - Tree broadcast - * - Rect broadcast - * ? Binomial broadcast - */ -/* ****************************************************************** */ -int MPIDO_Allgatherv_bcast(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - int buffer_sum, - const int *displs, - MPI_Datatype recvtype, - MPI_Aint send_true_lb, - MPI_Aint recv_true_lb, - size_t send_size, - size_t recv_size, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - const int rank = comm_ptr->rank; - TRACE_ERR("Entering MPIDO_Allgatherv_bcast\n"); - int i, rc=MPI_ERR_INTERN; - MPI_Aint extent; - MPIDU_Datatype_get_extent_macro(recvtype, extent); - - if (sendbuf != MPI_IN_PLACE) - { - void *destbuffer = recvbuf + displs[rank] * extent; - MPIR_Localcopy(sendbuf, - sendcount, - sendtype, - destbuffer, - recvcounts[rank], - recvtype); - } - - TRACE_ERR("Calling MPIDO_Bcasts in MPIDO_Allgatherv_bcast\n"); - for (i = 0; i < comm_ptr->local_size; i++) - { - void *destbuffer = recvbuf + displs[i] * extent; - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Bcast(destbuffer, - recvcounts[i], - recvtype, - i, - comm_ptr, - mpierrno); - } - TRACE_ERR("Leaving MPIDO_Allgatherv_bcast\n"); - - return rc; -} - -/* ****************************************************************** */ -/** - * \brief Use (tree/rect) MPIDO_Alltoall() to do a fast Allgatherv operation - * - * \note This function requires that: - * - The send/recv data types are contiguous - * - DMA alltoallv is availible (for max performance) - */ -/* ****************************************************************** */ - -int MPIDO_Allgatherv_alltoall(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int *recvcounts, - int buffer_sum, - const int *displs, - MPI_Datatype recvtype, - MPI_Aint send_true_lb, - MPI_Aint recv_true_lb, - size_t send_size, - size_t recv_size, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - TRACE_ERR("Entering MPIDO_Allgatherv_alltoallv\n"); - size_t total_send_size; - char *startbuf; - char *destbuf; - int i, rc; - int my_recvcounts = -1; - void *a2a_sendbuf = NULL; - const int size = comm_ptr->local_size; - int a2a_sendcounts[size]; - int a2a_senddispls[size]; - const int rank = comm_ptr->rank; - - total_send_size = recvcounts[rank] * recv_size; - for (i = 0; i < size; ++i) - { - a2a_sendcounts[i] = total_send_size; - a2a_senddispls[i] = 0; - } - if (sendbuf != MPI_IN_PLACE) - { - a2a_sendbuf = (char *)sendbuf + send_true_lb; - } - else - { - startbuf = (char *) recvbuf + recv_true_lb; - destbuf = startbuf + displs[rank] * recv_size; - a2a_sendbuf = destbuf; - a2a_sendcounts[rank] = 0; - my_recvcounts = recvcounts[rank]; - recvcounts[rank] = 0; - } - - TRACE_ERR("Calling alltoallv in MPIDO_Allgatherv_alltoallv\n"); - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Alltoallv(a2a_sendbuf, - a2a_sendcounts, - a2a_senddispls, - MPI_CHAR, - recvbuf, - recvcounts, - displs, - recvtype, - comm_ptr, - mpierrno); - if (sendbuf == MPI_IN_PLACE) - recvcounts[rank] = my_recvcounts; - - TRACE_ERR("Leaving MPIDO_Allgatherv_alltoallv\n"); - return rc; -} - - -int -MPIDO_Allgatherv(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *displs, - MPI_Datatype recvtype, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Allgatherv\n"); - /* function pointer to be used to point to approperiate algorithm */ - - /* Check the nature of the buffers */ - MPIDU_Datatype*dt_null = NULL; - MPI_Aint send_true_lb = 0; - MPI_Aint recv_true_lb = 0; - size_t send_size = 0; - size_t recv_size = 0; - int config[6]; - int scount=sendcount; - - int i, rc, buffer_sum = 0; - const int size = comm_ptr->local_size; - char use_tree_reduce, use_alltoall, use_bcast, use_pami, use_opt; - char *sbuf, *rbuf; - const int rank = comm_ptr->rank; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - int queryreq = 0; - -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const int selected_type = mpid->user_selected_type[PAMI_XFER_ALLGATHERV_INT]; - - pami_xfer_t allred; - volatile unsigned allred_active = 1; - volatile unsigned allgatherv_active = 1; - pami_type_t stype, rtype; - int tmp; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - - for(i=0;i<6;i++) config[i] = 1; - - allred.cb_done = allred_cb_done; - allred.cookie = (void *)&allred_active; - allred.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLREDUCE][0][0]; - allred.cmd.xfer_allreduce.sndbuf = (void *)config; - allred.cmd.xfer_allreduce.stype = PAMI_TYPE_SIGNED_INT; - allred.cmd.xfer_allreduce.rcvbuf = (void *)config; - allred.cmd.xfer_allreduce.rtype = PAMI_TYPE_SIGNED_INT; - allred.cmd.xfer_allreduce.stypecount = 6; - allred.cmd.xfer_allreduce.rtypecount = 6; - allred.cmd.xfer_allreduce.op = PAMI_DATA_BAND; - - use_alltoall = mpid->allgathervs[2]; - use_tree_reduce = mpid->allgathervs[0]; - use_bcast = mpid->allgathervs[1]; - use_pami = selected_type != MPID_COLL_USE_MPICH; - - if((sendbuf != MPI_IN_PLACE) && (MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp) != MPI_SUCCESS)) - use_pami = 0; - if(MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp) != MPI_SUCCESS) - use_pami = 0; - - use_opt = use_alltoall || use_tree_reduce || use_bcast || use_pami; - - if(!use_opt) /* back to MPICH */ - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH allgatherv type %u.\n", - selected_type); - TRACE_ERR("Using MPICH Allgatherv\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHERV_MPICH"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(sdt_extent * sendcount); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, sdt_extent * sendcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - size_t rtotal_buf; - if(is_recv_dev_buf) - { - //Since displs can be non-continous, we need to calculate max buffer size - int highest_displs = displs[size - 1]; - int highest_recvcount = recvcounts[size - 1]; - for(i = 0; i < size; i++) - { - if(displs[i]+recvcounts[i] > highest_displs+highest_recvcount) - { - highest_displs = displs[i]; - highest_recvcount = recvcounts[i]; - } - } - rtotal_buf = (highest_displs+highest_recvcount)*rdt_extent; - rcbuf = MPL_malloc(rtotal_buf); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, rtotal_buf, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, rtotal_buf); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Allgatherv(scbuf, sendcount, sendtype, rcbuf, recvcounts, displs, recvtype, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, rtotal_buf, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Allgatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - comm_ptr, mpierrno); - } - - MPIDI_Datatype_get_info(1, - recvtype, - config[MPID_RECV_CONTIG], - recv_size, - dt_null, - recv_true_lb); - - if(sendbuf == MPI_IN_PLACE) - { - sbuf = PAMI_IN_PLACE; - if(unlikely(verbose)) - fprintf(stderr,"allgatherv MPI_IN_PLACE buffering\n"); - stype = rtype; - scount = recvcounts[rank]; - send_size = recv_size * scount; - } - else - { - MPIDI_Datatype_get_info(sendcount, - sendtype, - config[MPID_SEND_CONTIG], - send_size, - dt_null, - send_true_lb); - sbuf = (char *)sendbuf+send_true_lb; - } - - rbuf = (char *)recvbuf+recv_true_lb; - - if(use_alltoall || use_bcast || use_tree_reduce) - { - if (displs[0]) - config[MPID_RECV_CONTINUOUS] = 0; - - for (i = 1; i < size; i++) - { - buffer_sum += recvcounts[i - 1]; - if (buffer_sum != displs[i]) - { - config[MPID_RECV_CONTINUOUS] = 0; - break; - } - } - - buffer_sum += recvcounts[size - 1]; - - buffer_sum *= recv_size; - - /* disable with "safe allgatherv" env var */ - if(mpid->preallreduces[MPID_ALLGATHERV_PREALLREDUCE]) - { - MPIDI_Post_coll_t allred_post; - MPIDI_Context_post(MPIDI_Context[0], &allred_post.state, - MPIDI_Pami_post_wrapper, (void *)&allred); - - MPID_PROGRESS_WAIT_WHILE(allred_active); - } - - use_tree_reduce = mpid->allgathervs[0] && - config[MPID_RECV_CONTIG] && config[MPID_SEND_CONTIG] && - config[MPID_RECV_CONTINUOUS] && buffer_sum % sizeof(unsigned) == 0; - - use_alltoall = mpid->allgathervs[2] && - config[MPID_RECV_CONTIG] && config[MPID_SEND_CONTIG]; - - use_bcast = mpid->allgathervs[1]; - } - - if(use_pami) - { - pami_xfer_t allgatherv; - allgatherv.cb_done = allgatherv_cb_done; - allgatherv.cookie = (void *)&allgatherv_active; - if(selected_type == MPID_COLL_OPTIMIZED) - { - if((mpid->cutoff_size[PAMI_XFER_ALLGATHERV_INT][0] == 0) || - (mpid->cutoff_size[PAMI_XFER_ALLGATHERV_INT][0] > 0 && mpid->cutoff_size[PAMI_XFER_ALLGATHERV_INT][0] >= send_size)) - { - allgatherv.algorithm = mpid->opt_protocol[PAMI_XFER_ALLGATHERV_INT][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLGATHERV_INT][0]; - queryreq = mpid->must_query[PAMI_XFER_ALLGATHERV_INT][0]; - } - else - return MPIR_Allgatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - comm_ptr, mpierrno); - } - else - { - allgatherv.algorithm = mpid->user_selected[PAMI_XFER_ALLGATHERV_INT]; - my_md = &mpid->user_metadata[PAMI_XFER_ALLGATHERV_INT]; - queryreq = selected_type; - } - - allgatherv.cmd.xfer_allgatherv_int.sndbuf = sbuf; - allgatherv.cmd.xfer_allgatherv_int.rcvbuf = rbuf; - - allgatherv.cmd.xfer_allgatherv_int.stype = stype; - allgatherv.cmd.xfer_allgatherv_int.rtype = rtype; - allgatherv.cmd.xfer_allgatherv_int.stypecount = scount; - allgatherv.cmd.xfer_allgatherv_int.rtypecounts = (int *) recvcounts; - allgatherv.cmd.xfer_allgatherv_int.rdispls = (int *) displs; - - if(unlikely (queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("Querying allgatherv_int protocol %s, type was %d\n", my_md->name, - selected_type); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; -/* Can't check ranges like this. Non-local. Comment out for now. - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb; - MPIDU_Datatype*data_ptr; - int data_size, data_contig; - MPIDI_Datatype_get_info(sendcount, sendtype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - */ - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&allgatherv); - TRACE_ERR("Allgatherv bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH allgatherv.\n", my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHERV_MPICH"); - return MPIR_Allgatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for allgatherv on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - MPIDI_Post_coll_t allgatherv_post; - MPIDI_Context_post(MPIDI_Context[0], &allgatherv_post.state, - MPIDI_Pami_post_wrapper, (void *)&allgatherv); - - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - - TRACE_ERR("Rank %d waiting on active %d\n", rank, allgatherv_active); - MPID_PROGRESS_WAIT_WHILE(allgatherv_active); - - return PAMI_SUCCESS; - } - - /* TODO These need ordered in speed-order */ - if(use_tree_reduce) - { - if(unlikely(verbose)) - fprintf(stderr,"Using tree reduce allgatherv type %u.\n", - selected_type); - rc = MPIDO_Allgatherv_allreduce(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, buffer_sum, displs, recvtype, - send_true_lb, recv_true_lb, send_size, recv_size, - comm_ptr, mpierrno); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHERV_OPT_ALLREDUCE"); - return rc; - } - - if(use_bcast) - { - if(unlikely(verbose)) - fprintf(stderr,"Using bcast allgatherv type %u.\n", - selected_type); - rc = MPIDO_Allgatherv_bcast(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, buffer_sum, displs, recvtype, - send_true_lb, recv_true_lb, send_size, recv_size, - comm_ptr, mpierrno); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHERV_OPT_BCAST"); - return rc; - } - - if(use_alltoall) - { - if(unlikely(verbose)) - fprintf(stderr,"Using alltoall allgatherv type %u.\n", - selected_type); - rc = MPIDO_Allgatherv_alltoall(sendbuf, sendcount, sendtype, - recvbuf, (int *)recvcounts, buffer_sum, displs, recvtype, - send_true_lb, recv_true_lb, send_size, recv_size, - comm_ptr, mpierrno); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHERV_OPT_ALLTOALL"); - return rc; - } - - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH allgatherv type %u.\n", - selected_type); - TRACE_ERR("Using MPICH for Allgatherv\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLGATHERV_MPICH"); - return MPIR_Allgatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - comm_ptr, mpierrno); -} - -int -MPIDO_Allgatherv_simple(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *displs, - MPI_Datatype recvtype, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Allgatherv_optimized\n"); - /* function pointer to be used to point to approperiate algorithm */ - /* Check the nature of the buffers */ - MPIDU_Datatype*dt_null = NULL; - MPI_Aint send_true_lb = 0; - MPI_Aint recv_true_lb = 0; - size_t send_size = 0; - size_t recv_size = 0; - size_t rcvtypelen = 0; - int snd_data_contig = 0, rcv_data_contig = 0; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - int scount=sendcount; - - char *sbuf, *rbuf; - pami_type_t stype = NULL, rtype; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - int recvcontinuous=0; - size_t totalrecvcount=0; - int *lrecvdispls = NULL; /* possible local displs calculated for noncontinous */ - int *lrecvcounts = NULL;/* possible local counts calculated for noncontinous */ - const int *precvdispls = displs; /* pointer to displs to use as pami parmi */ - const int *precvcounts = recvcounts; /* pointer to counts to use as pami parmi */ - int inplace = sendbuf == MPI_IN_PLACE? 1 : 0; - - - volatile unsigned allgatherv_active = 1; - int recvok=PAMI_SUCCESS, sendok=PAMI_SUCCESS; - int tmp; - const pami_metadata_t *my_md; - - - MPIDI_Datatype_get_info(1, - recvtype, - rcv_data_contig, - rcvtypelen, - dt_null, - recv_true_lb); - - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_ALLGATHERV_INT, rcvtypelen * recvcounts[0], advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Allgatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - - if(!inplace) - { - sendok = MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp); - MPIDI_Datatype_get_info(sendcount, sendtype, snd_data_contig, send_size, dt_null, send_true_lb); - sbuf = (char *)sendbuf + send_true_lb; - if(!snd_data_contig || (sendok != PAMI_SUCCESS)) - { - stype = PAMI_TYPE_UNSIGNED_CHAR; - scount = send_size; - if(!snd_data_contig) - { - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - MPIR_Localcopy(sendbuf, sendcount, sendtype, - snd_noncontig_buff, send_size,MPI_CHAR); - } - } - } - else - sbuf = PAMI_IN_PLACE; - - recvok = MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp); - rbuf = (char *)recvbuf+recv_true_lb; - if(!rcv_data_contig || (recvok != PAMI_SUCCESS)) - { - rtype = PAMI_TYPE_UNSIGNED_CHAR; - totalrecvcount = recvcounts[0]; - recvcontinuous = displs[0] == 0? 1 : 0 ; - int i; - precvdispls = lrecvdispls = MPL_malloc(size*sizeof(int)); - precvcounts = lrecvcounts = MPL_malloc(size*sizeof(int)); - lrecvdispls[0]= 0; - lrecvcounts[0]= rcvtypelen * recvcounts[0]; - for(i=1; i<size; ++i) - { - lrecvdispls[i]= rcvtypelen * totalrecvcount; - totalrecvcount += recvcounts[i]; - if(displs[i] != (displs[i-1] + recvcounts[i-1])) - recvcontinuous = 0; - lrecvcounts[i]= rcvtypelen * recvcounts[i]; - } - recv_size = rcvtypelen * totalrecvcount; - TRACE_ERR("Pack receive rcv_contig %zu, recvok %zd, totalrecvcount %zu, recvcontinuous %zu, rcvtypelen %zu, recv_size %zu\n", - (size_t)rcv_data_contig, (size_t)recvok, (size_t)totalrecvcount, (size_t)recvcontinuous,(size_t)rcvtypelen, (size_t)recv_size); - - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(inplace) - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - MPIR_Localcopy(recvbuf + displs[rank]*extent, recvcounts[rank], recvtype, - rcv_noncontig_buff + precvdispls[rank], precvcounts[rank],MPI_CHAR); - scount = precvcounts[rank]; - stype = PAMI_TYPE_UNSIGNED_CHAR; - sbuf = PAMI_IN_PLACE; - } - } - - - pami_xfer_t allgatherv; - allgatherv.cb_done = allgatherv_cb_done; - allgatherv.cookie = (void *)&allgatherv_active; - allgatherv.cmd.xfer_allgatherv_int.sndbuf = sbuf; - allgatherv.cmd.xfer_allgatherv_int.rcvbuf = rbuf; - allgatherv.cmd.xfer_allgatherv_int.stype = stype;/* stype is ignored when sndbuf == PAMI_IN_PLACE */ - allgatherv.cmd.xfer_allgatherv_int.rtype = rtype; - allgatherv.cmd.xfer_allgatherv_int.stypecount = scount; - allgatherv.cmd.xfer_allgatherv_int.rtypecounts = (int *) precvcounts; - allgatherv.cmd.xfer_allgatherv_int.rdispls = (int *) precvdispls; - allgatherv.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLGATHERV_INT][0][0]; - my_md = &mpid->coll_metadata[PAMI_XFER_ALLGATHERV_INT][0][0]; - - TRACE_ERR("Calling allgatherv via %s()\n", MPIDI_Process.context_post.active>0?"PAMI_Collective":"PAMI_Context_post"); - MPIDI_Post_coll_t allgatherv_post; - MPIDI_Context_post(MPIDI_Context[0], &allgatherv_post.state, - MPIDI_Pami_post_wrapper, (void *)&allgatherv); - - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - - TRACE_ERR("Rank %d waiting on active %d\n", rank, allgatherv_active); - MPID_PROGRESS_WAIT_WHILE(allgatherv_active); - - if(!rcv_data_contig || (recvok != PAMI_SUCCESS)) - { - if(recvcontinuous) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size,MPI_CHAR, - recvbuf, totalrecvcount, recvtype); - } - else - { - size_t extent; - int i; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)rcv_noncontig_buff+ precvdispls[i]; - char* rcbuf = (char*)recvbuf + displs[i]*extent; - MPIR_Localcopy(scbuf, precvcounts[i], MPI_CHAR, - rcbuf, recvcounts[i], recvtype); - TRACE_ERR("Pack recv src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)precvdispls[i],(size_t)i,(size_t)precvcounts[i],(size_t)precvdispls[i], *(int*)scbuf); - TRACE_ERR("Pack recv dest extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)displs[i],(size_t)i,(size_t)recvcounts[i],(size_t)displs[i], *(int*)rcbuf); - } - } - MPL_free(rcv_noncontig_buff); - } - if(!snd_data_contig) MPL_free(snd_noncontig_buff); - if(lrecvdispls) MPL_free(lrecvdispls); - if(lrecvcounts) MPL_free(lrecvcounts); - - return MPI_SUCCESS; -} - - -int -MPIDO_CSWrapper_allgatherv(pami_xfer_t *allgatherv, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *sbuf; - MPIDI_coll_check_in_place(allgatherv->cmd.xfer_allgatherv_int.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( allgatherv->cmd.xfer_allgatherv_int.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( allgatherv->cmd.xfer_allgatherv_int.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Allgatherv(sbuf, - allgatherv->cmd.xfer_allgatherv_int.stypecount, sendtype, - allgatherv->cmd.xfer_allgatherv_int.rcvbuf, - allgatherv->cmd.xfer_allgatherv_int.rtypecounts, - allgatherv->cmd.xfer_allgatherv_int.rdispls, recvtype, - comm_ptr, &mpierrno); - if(allgatherv->cb_done && rc == 0) - allgatherv->cb_done(NULL, allgatherv->cookie, PAMI_SUCCESS); - return rc; - - -} - diff --git a/src/mpid/pamid/src/coll/allgatherv/mpido_iallgatherv.c b/src/mpid/pamid/src/coll/allgatherv/mpido_iallgatherv.c deleted file mode 100644 index b7570cd..0000000 --- a/src/mpid/pamid/src/coll/allgatherv/mpido_iallgatherv.c +++ /dev/null @@ -1,77 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/allgatherv/mpido_iallgatherv.c - * \brief ??? - */ - -/*#define TRACE_ON */ -#include <mpidimpl.h> - -int -MPIDO_Iallgatherv(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *displs, - MPI_Datatype recvtype, - MPIR_Comm * comm_ptr, - MPIR_Request ** request) -{ - TRACE_ERR("Entering MPIDO_Iallgatherv\n"); - - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iallgather_impl() function to invoke the mpich nbc implementation - * of MPI_Iallgather(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iallgather() with a blocking MPIR_Allgather(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH iallgather algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Allgatherv_impl(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - comm_ptr, &mpierrno); - - /* - * The blocking allgather has completed - create and complete a - * MPIR_Request object so the MPIR_Iallgather_impl() function does not - * perform an additional iallgather. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/allreduce/Makefile.mk b/src/mpid/pamid/src/coll/allreduce/Makefile.mk deleted file mode 100644 index 875e2d9..0000000 --- a/src/mpid/pamid/src/coll/allreduce/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/allreduce/mpido_allreduce.c \ - src/mpid/pamid/src/coll/allreduce/mpido_iallreduce.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/allreduce/mpido_allreduce.c b/src/mpid/pamid/src/coll/allreduce/mpido_allreduce.c deleted file mode 100644 index 48f0bfc..0000000 --- a/src/mpid/pamid/src/coll/allreduce/mpido_allreduce.c +++ /dev/null @@ -1,562 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/allreduce/mpido_allreduce.c - * \brief ??? - */ - -/* #define TRACE_ON */ - -#include <mpidimpl.h> -/* -#undef TRACE_ERR -#define TRACE_ERR(format, ...) fprintf(stderr, format, ##__VA_ARGS__) -*/ -static void cb_allreduce(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *) clientdata; - TRACE_ERR("callback enter, active: %d\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - -int MPIDO_Allreduce(const void *sendbuf, - void *recvbuf, - int count, - MPI_Datatype dt, - MPI_Op op, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - void *sbuf; - TRACE_ERR("Entering mpido_allreduce\n"); - pami_type_t pdt; - pami_data_function pop; - int mu; - int rc; -#ifdef TRACE_ON - int len; - char op_str[255]; - char dt_str[255]; - MPIDI_Op_to_string(op, op_str); - PMPI_Type_get_name(dt, dt_str, &len); -#endif - volatile unsigned active = 1; - pami_xfer_t allred; - pami_algorithm_t my_allred = 0; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int alg_selected = 0; - const int rank = comm_ptr->rank; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_ALLREDUCE]; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - int queryreq = 0; - if(likely(dt == MPI_DOUBLE || dt == MPI_DOUBLE_PRECISION)) - { - rc = MPI_SUCCESS; - pdt = PAMI_TYPE_DOUBLE; - if(likely(selected_type == MPID_COLL_OPTIMIZED) && - (mpid->query_cached_allreduce != MPID_COLL_USE_MPICH)) - { - /* double protocol works on all message sizes */ - my_allred = mpid->cached_allreduce; - my_md = &mpid->cached_allreduce_md; - alg_selected = 1; - } - if(likely(op == MPI_SUM)) - pop = PAMI_DATA_SUM; - else if(likely(op == MPI_MAX)) - pop = PAMI_DATA_MAX; - else if(likely(op == MPI_MIN)) - pop = PAMI_DATA_MIN; - else - { - alg_selected = 0; - rc = MPIDI_Datatype_to_pami(dt, &pdt, op, &pop, &mu); - } - } - else rc = MPIDI_Datatype_to_pami(dt, &pdt, op, &pop, &mu); - - if(unlikely(verbose)) - fprintf(stderr,"allred rc %u,count %d, Datatype %p, op %p, mu %u, selectedvar %u != %u, sendbuf %p, recvbuf %p\n", - rc, count, pdt, pop, mu, - (unsigned)selected_type,MPID_COLL_USE_MPICH, sendbuf, recvbuf); - /* convert to metadata query */ - /* Punt count 0 allreduce to MPICH. Let them do whatever's 'right' */ - if(unlikely(rc != MPI_SUCCESS || (count==0) || - selected_type == MPID_COLL_USE_MPICH)) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH allreduce type %u.\n", - selected_type); - MPIDI_Update_last_algorithm(comm_ptr, "ALLREDUCE_MPICH"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint dt_extent; - MPIDU_Datatype_get_extent_macro(dt, dt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(dt_extent * count); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, dt_extent * count, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(dt_extent * count); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, dt_extent * count, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, dt_extent * count); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Allreduce(scbuf, rcbuf, count, dt, op, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, dt_extent * count, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Allreduce(sendbuf, recvbuf, count, dt, op, comm_ptr, mpierrno); - } - - sbuf = (void *)sendbuf; - if(unlikely(sendbuf == MPI_IN_PLACE)) - { - if(unlikely(verbose)) - fprintf(stderr,"allreduce MPI_IN_PLACE buffering\n"); - sbuf = PAMI_IN_PLACE; - } - - allred.cb_done = cb_allreduce; - allred.cookie = (void *)&active; - allred.cmd.xfer_allreduce.sndbuf = sbuf; - allred.cmd.xfer_allreduce.stype = pdt; - allred.cmd.xfer_allreduce.rcvbuf = recvbuf; - allred.cmd.xfer_allreduce.rtype = pdt; - allred.cmd.xfer_allreduce.stypecount = count; - allred.cmd.xfer_allreduce.rtypecount = count; - allred.cmd.xfer_allreduce.op = pop; - - TRACE_ERR("Allreduce - Basic Collective Selection\n"); - - if(unlikely(!alg_selected)) /* Cached double algorithm not selected above */ - { - if(likely(selected_type == MPID_COLL_OPTIMIZED)) - { - if(mpid->query_cached_allreduce != MPID_COLL_USE_MPICH) - { /* try the cached algorithm first, assume it's always a query algorithm so query now */ - my_allred = mpid->cached_allreduce; - my_md = &mpid->cached_allreduce_md; - alg_selected = 1; - if(my_md->check_fn != NULL)/*This should always be the case in FCA.. Otherwise punt to mpich*/ - { - metadata_result_t result = {0}; - TRACE_ERR("querying allreduce algorithm %s\n", - my_md->name); - result = my_md->check_fn(&allred); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - /* \todo Ignore check_correct.values.nonlocal until we implement the - 'pre-allreduce allreduce' or the 'safe' environment flag. - We will basically assume 'safe' -- that all ranks are aligned (or not). - */ - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(!result.bitmask) - ; /* ok, algorithm selected */ - else - { - alg_selected = 0; - if(unlikely(verbose)) - fprintf(stderr,"check_fn failed for %s.\n", my_md->name); - } - } - else /* no check_fn, manually look at the metadata fields */ - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - /* Check if the message range if restricted */ - if(my_md->check_correct.values.rangeminmax) - { - int data_size; - MPIDI_Datatype_get_data_size(count, dt, data_size); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - if(unlikely(verbose)) - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - alg_selected = 0; - } - } - /* \todo check the rest of the metadata */ - } - } - /* If we didn't use the cached protocol above (query failed?) then check regular optimized protocol fields */ - if(!alg_selected) - { - const int queryreq0 = mpid->must_query[PAMI_XFER_ALLREDUCE][0]; - const int queryreq1 = mpid->must_query[PAMI_XFER_ALLREDUCE][1]; - /* TODO this really needs to be cleaned up for BGQ and fca */ - if(queryreq0 == MPID_COLL_NOQUERY && - count <= mpid->cutoff_size[PAMI_XFER_ALLREDUCE][0]) - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - my_allred = mpid->opt_protocol[PAMI_XFER_ALLREDUCE][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLREDUCE][0]; - alg_selected = 1; - } - else if(queryreq1 == MPID_COLL_NOQUERY && - count > mpid->cutoff_size[PAMI_XFER_ALLREDUCE][0]) - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - my_allred = mpid->opt_protocol[PAMI_XFER_ALLREDUCE][1]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLREDUCE][1]; - alg_selected = 1; - } - else if(((queryreq0 == MPID_COLL_CHECK_FN_REQUIRED) || - (queryreq0 == MPID_COLL_QUERY) || - (queryreq0 == MPID_COLL_ALWAYS_QUERY)) && - ((mpid->cutoff_size[PAMI_XFER_ALLREDUCE][0] == 0) || - (count <= mpid->cutoff_size[PAMI_XFER_ALLREDUCE][0] && mpid->cutoff_size[PAMI_XFER_ALLREDUCE][0] > 0))) - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - my_allred = mpid->opt_protocol[PAMI_XFER_ALLREDUCE][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLREDUCE][0]; - alg_selected = 1; - queryreq = queryreq0; - } - else if((queryreq1 == MPID_COLL_CHECK_FN_REQUIRED) || - (queryreq1 == MPID_COLL_QUERY) || - (queryreq1 == MPID_COLL_ALWAYS_QUERY)) - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - my_allred = mpid->opt_protocol[PAMI_XFER_ALLREDUCE][1]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLREDUCE][1]; - alg_selected = 1; - queryreq = queryreq1; - } - TRACE_ERR("Alg selected: %d\n", alg_selected); - if(likely(alg_selected)) - { - if(unlikely((queryreq == MPID_COLL_CHECK_FN_REQUIRED) || - (queryreq == MPID_COLL_QUERY) || - (queryreq == MPID_COLL_ALWAYS_QUERY))) - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - if(my_md->check_fn != NULL)/*This should always be the case in FCA.. Otherwise punt to mpich*/ - { - metadata_result_t result = {0}; - TRACE_ERR("querying allreduce algorithm %s\n", - my_md->name); - result = my_md->check_fn(&allred); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - /* \todo Ignore check_correct.values.nonlocal until we implement the - 'pre-allreduce allreduce' or the 'safe' environment flag. - We will basically assume 'safe' -- that all ranks are aligned (or not). - */ - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(!result.bitmask) - ; /* ok, algorithm selected */ - else - { - alg_selected = 0; - if(unlikely(verbose)) - fprintf(stderr,"check_fn failed for %s.\n", my_md->name); - } - } - else /* no check_fn, manually look at the metadata fields */ - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - /* Check if the message range if restricted */ - if(my_md->check_correct.values.rangeminmax) - { - int data_size; - MPIDI_Datatype_get_data_size(count, dt, data_size); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - if(unlikely(verbose)) - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - alg_selected = 0; - } - } - /* \todo check the rest of the metadata */ - } - } - else - { - TRACE_ERR("Using %s for allreduce\n", my_md->name); - } - } - } - } - else - { - TRACE_ERR("Non-Optimzed selection line %d\n",__LINE__); - my_allred = mpid->user_selected[PAMI_XFER_ALLREDUCE]; - my_md = &mpid->user_metadata[PAMI_XFER_ALLREDUCE]; - if(selected_type == MPID_COLL_QUERY || - selected_type == MPID_COLL_ALWAYS_QUERY || - selected_type == MPID_COLL_CHECK_FN_REQUIRED) - { - TRACE_ERR("Non-Optimzed selection line %d\n",__LINE__); - if(my_md->check_fn != NULL) - { - /* For now, we don't distinguish between MPID_COLL_ALWAYS_QUERY & - MPID_COLL_CHECK_FN_REQUIRED, we just call the fn */ - metadata_result_t result = {0}; - TRACE_ERR("querying allreduce algorithm %s, type was %d\n", - my_md->name, - selected_type); - result = mpid->user_metadata[PAMI_XFER_ALLREDUCE].check_fn(&allred); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - /* \todo Ignore check_correct.values.nonlocal until we implement the - 'pre-allreduce allreduce' or the 'safe' environment flag. - We will basically assume 'safe' -- that all ranks are aligned (or not). - */ - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(!result.bitmask) - alg_selected = 1; /* query algorithm successfully selected */ - else - if(unlikely(verbose)) - fprintf(stderr,"check_fn failed for %s.\n", my_md->name); - } - else /* no check_fn, manually look at the metadata fields */ - { - TRACE_ERR("Non-Optimzed selection line %d\n",__LINE__); - /* Check if the message range if restricted */ - if(my_md->check_correct.values.rangeminmax) - { - int data_size; - MPIDI_Datatype_get_data_size(count, dt, data_size); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - alg_selected = 1; /* query algorithm successfully selected */ - else - if(unlikely(verbose)) - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - /* \todo check the rest of the metadata */ - } - } - else alg_selected = 1; /* non-query algorithm selected */ - - } - } - - if(unlikely(!alg_selected)) /* must be fallback to MPICH */ - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH allreduce\n"); - MPIDI_Update_last_algorithm(comm_ptr, "ALLREDUCE_MPICH"); - return MPIR_Allreduce(sendbuf, recvbuf, count, dt, op, comm_ptr, mpierrno); - } - allred.algorithm = my_allred; - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for allreduce on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - - MPIDI_Post_coll_t allred_post; - MPIDI_Context_post(MPIDI_Context[0], &allred_post.state, - MPIDI_Pami_post_wrapper, (void *)&allred); - - MPID_assert(rc == PAMI_SUCCESS); - MPIDI_Update_last_algorithm(comm_ptr,my_md->name); - MPID_PROGRESS_WAIT_WHILE(active); - TRACE_ERR("allreduce done\n"); - return MPI_SUCCESS; -} - -int MPIDO_Allreduce_simple(const void *sendbuf, - void *recvbuf, - int count, - MPI_Datatype dt, - MPI_Op op, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - void *sbuf; - TRACE_ERR("Entering MPIDO_Allreduce_optimized\n"); - pami_type_t pdt; - pami_data_function pop; - int mu; - int rc; -#ifdef TRACE_ON - int len; - char op_str[255]; - char dt_str[255]; - MPIDI_Op_to_string(op, op_str); - PMPI_Type_get_name(dt, dt_str, &len); -#endif - volatile unsigned active = 1; - pami_xfer_t allred; - const pami_metadata_t *my_allred_md = (pami_metadata_t *)NULL; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - MPIDU_Datatype*data_ptr; - MPI_Aint data_true_lb = 0; - int data_size, data_contig; - - MPIDI_Datatype_get_info(1, dt, - data_contig, data_size, data_ptr, data_true_lb); - - - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_ALLREDUCE, data_size * count, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Allreduce(sendbuf, recvbuf, count, dt, op, comm_ptr, mpierrno); - } - } - } - - - rc = MPIDI_Datatype_to_pami(dt, &pdt, op, &pop, &mu); - - /* convert to metadata query */ - /* Punt count 0 allreduce to MPICH. Let them do whatever's 'right' */ - if(unlikely(rc != MPI_SUCCESS || (count==0))) - { - MPIDI_Update_last_algorithm(comm_ptr, "ALLREDUCE_MPICH"); - return MPIR_Allreduce(sendbuf, recvbuf, count, dt, op, comm_ptr, mpierrno); - } - - if(!data_contig) - { - MPIDI_Update_last_algorithm(comm_ptr, "ALLREDUCE_MPICH"); - return MPIR_Allreduce(sendbuf, recvbuf, count, dt, op, comm_ptr, mpierrno); - } - - sbuf = (void *)sendbuf; - if(unlikely(sendbuf == MPI_IN_PLACE)) - { - sbuf = PAMI_IN_PLACE; - } - - allred.cb_done = cb_allreduce; - allred.cookie = (void *)&active; - allred.cmd.xfer_allreduce.sndbuf = sbuf; - allred.cmd.xfer_allreduce.stype = pdt; - allred.cmd.xfer_allreduce.rcvbuf = recvbuf; - allred.cmd.xfer_allreduce.rtype = pdt; - allred.cmd.xfer_allreduce.stypecount = count; - allred.cmd.xfer_allreduce.rtypecount = count; - allred.cmd.xfer_allreduce.op = pop; - allred.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLREDUCE][0][0]; - my_allred_md = &mpid->coll_metadata[PAMI_XFER_ALLREDUCE][0][0]; - - MPIDI_Post_coll_t allred_post; - MPIDI_Context_post(MPIDI_Context[0], &allred_post.state, - MPIDI_Pami_post_wrapper, (void *)&allred); - - MPID_assert(rc == PAMI_SUCCESS); - MPIDI_Update_last_algorithm(comm_ptr,my_allred_md->name); - MPID_PROGRESS_WAIT_WHILE(active); - TRACE_ERR("Allreduce done\n"); - return MPI_SUCCESS; -} - -int -MPIDO_CSWrapper_allreduce(pami_xfer_t *allreduce, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype type; - MPI_Op op; - void *sbuf; - MPIDI_coll_check_in_place(allreduce->cmd.xfer_allreduce.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( allreduce->cmd.xfer_allreduce.stype, - &type, - allreduce->cmd.xfer_allreduce.op, - &op); - if(rc == -1) return rc; - - rc = MPIR_Allreduce(sbuf, - allreduce->cmd.xfer_allreduce.rcvbuf, - allreduce->cmd.xfer_allreduce.rtypecount, - type, op, comm_ptr, &mpierrno); - if(allreduce->cb_done && rc == 0) - allreduce->cb_done(NULL, allreduce->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/allreduce/mpido_iallreduce.c b/src/mpid/pamid/src/coll/allreduce/mpido_iallreduce.c deleted file mode 100644 index d504b80..0000000 --- a/src/mpid/pamid/src/coll/allreduce/mpido_iallreduce.c +++ /dev/null @@ -1,73 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/allreduce/mpido_iallreduce.c - * \brief ??? - */ - -/*#define TRACE_ON*/ - -#include <mpidimpl.h> - -int MPIDO_Iallreduce(const void *sendbuf, - void *recvbuf, - int count, - MPI_Datatype dt, - MPI_Op op, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - TRACE_ERR("Entering mpido_iallreduce\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iallreduce_impl() function to invoke the mpich nbc implementation - * of MPI_Iallreduce(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iallreduce() with a blocking MPIR_Allreduce(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH iallreduce algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Allreduce_impl(sendbuf, recvbuf, count, dt, op, - comm_ptr, &mpierrno); - - /* - * The blocking allreduce has completed - create and complete a - * MPIR_Request object so the MPIR_Iallreduce_impl() function does not - * perform an additional iallreduce. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/alltoall/Makefile.mk b/src/mpid/pamid/src/coll/alltoall/Makefile.mk deleted file mode 100644 index f82f25e..0000000 --- a/src/mpid/pamid/src/coll/alltoall/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/alltoall/mpido_alltoall.c \ - src/mpid/pamid/src/coll/alltoall/mpido_ialltoall.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/alltoall/mpido_alltoall.c b/src/mpid/pamid/src/coll/alltoall/mpido_alltoall.c deleted file mode 100644 index a3354ae..0000000 --- a/src/mpid/pamid/src/coll/alltoall/mpido_alltoall.c +++ /dev/null @@ -1,449 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/alltoall/mpido_alltoall.c - * \brief ??? - */ - -/* #define TRACE_ON */ - -#include <mpidimpl.h> - -static void cb_alltoall(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *)clientdata; - TRACE_ERR("alltoall callback enter, active: %d\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - - -int MPIDO_Alltoall(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Alltoall\n"); - volatile unsigned active = 1; - MPIDU_Datatype*sdt, *rdt; - pami_type_t stype, rtype; - MPI_Aint sdt_true_lb=0, rdt_true_lb; - MPIDI_Post_coll_t alltoall_post; - int snd_contig, rcv_contig, pamidt=1; - int sndlen ATTRIBUTE((unused)), rcvlen ATTRIBUTE((unused)); - int tmp; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (comm_ptr->rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_ALLTOALL]; - - if(sendbuf != MPI_IN_PLACE) - { - MPIDI_Datatype_get_info(1, sendtype, snd_contig, sndlen, sdt, sdt_true_lb); - if(!snd_contig) pamidt = 0; - } - MPIDI_Datatype_get_info(1, recvtype, rcv_contig, rcvlen, rdt, rdt_true_lb); - if(!rcv_contig) pamidt = 0; - - /* Alltoall is much simpler if bytes are required because we don't need to - * malloc displ/count arrays and copy things - */ - - - /* Is it a built in type? If not, send to MPICH */ - if(sendbuf != MPI_IN_PLACE && (MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp) != MPI_SUCCESS)) - pamidt = 0; - if(MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp) != MPI_SUCCESS) - pamidt = 0; - - if((selected_type == MPID_COLL_USE_MPICH) || - pamidt == 0) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH alltoall algorithm\n"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(sdt_extent * sendcount); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, sdt_extent * sendcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(recvcount * rdt_extent); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, recvcount * rdt_extent, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, recvcount * rdt_extent); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Alltoall_intra(scbuf, sendcount, sendtype, rcbuf, recvcount, recvtype, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, recvcount * rdt_extent, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Alltoall_intra(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - - } - - pami_xfer_t alltoall; - pami_algorithm_t my_alltoall; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized alltoall was pre-selected\n"); - my_alltoall = mpid->opt_protocol[PAMI_XFER_ALLTOALL][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLTOALL][0]; - queryreq = mpid->must_query[PAMI_XFER_ALLTOALL][0]; - } - else - { - TRACE_ERR("Alltoall was specified by user\n"); - my_alltoall = mpid->user_selected[PAMI_XFER_ALLTOALL]; - my_md = &mpid->user_metadata[PAMI_XFER_ALLTOALL]; - queryreq = selected_type; - } - char *pname = my_md->name; - TRACE_ERR("Using alltoall protocol %s\n", pname); - - alltoall.cb_done = cb_alltoall; - alltoall.cookie = (void *)&active; - alltoall.algorithm = my_alltoall; - if(sendbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"alltoall MPI_IN_PLACE buffering\n"); - alltoall.cmd.xfer_alltoall.stype = rtype; - alltoall.cmd.xfer_alltoall.stypecount = recvcount; - alltoall.cmd.xfer_alltoall.sndbuf = PAMI_IN_PLACE; - } - else - { - alltoall.cmd.xfer_alltoall.stype = stype; - alltoall.cmd.xfer_alltoall.stypecount = sendcount; - alltoall.cmd.xfer_alltoall.sndbuf = (char *)sendbuf + sdt_true_lb; - } - alltoall.cmd.xfer_alltoall.rcvbuf = (char *)recvbuf + rdt_true_lb; - - alltoall.cmd.xfer_alltoall.rtypecount = recvcount; - alltoall.cmd.xfer_alltoall.rtype = rtype; - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying alltoall protocol %s, query level was %d\n", pname, - queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb ATTRIBUTE((unused)); - MPIDU_Datatype*data_ptr; - int data_size, data_contig ATTRIBUTE((unused)); - MPIDI_Datatype_get_info(sendcount, sendtype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&alltoall); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH alltoall.\n", pname); - MPIDI_Update_last_algorithm(comm_ptr, "ALLTOALL_MPICH"); - return MPIR_Alltoall_intra(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for alltoall on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - - MPIDI_Context_post(MPIDI_Context[0], &alltoall_post.state, - MPIDI_Pami_post_wrapper, (void *)&alltoall); - - TRACE_ERR("Waiting on active\n"); - MPID_PROGRESS_WAIT_WHILE(active); - - TRACE_ERR("Leaving alltoall\n"); - return PAMI_SUCCESS; -} - - -int MPIDO_Alltoall_simple(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Alltoall_optimized\n"); - volatile unsigned active = 1; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - size_t send_size = 0; - size_t recv_size = 0; - MPIDU_Segment segment; - MPIDU_Datatype*sdt, *rdt; - MPI_Aint sdt_true_lb=0, rdt_true_lb; - MPIDI_Post_coll_t alltoall_post; - int sndlen, rcvlen, snd_contig = 1, rcv_contig = 1; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (comm_ptr->rank == 0); -#endif - - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - - MPIDI_Datatype_get_info(1, recvtype, rcv_contig, rcvlen, rdt, rdt_true_lb); - rbuf = (char *)recvbuf + rdt_true_lb; - recv_size = rcvlen * recvcount; - - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_ALLTOALL, recv_size, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Alltoall_intra(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - if(sendbuf != MPI_IN_PLACE) - { - MPIDI_Datatype_get_info(1, sendtype, snd_contig, sndlen, sdt, sdt_true_lb); - sbuf = (char *)sendbuf + sdt_true_lb; - send_size = sndlen * sendcount; - if(!snd_contig) - { - snd_noncontig_buff = MPL_malloc(send_size*size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - DLOOP_Offset last = send_size*size; - MPIDU_Segment_init(sendbuf, sendcount*size, sendtype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, snd_noncontig_buff); - - } - } - - if(!rcv_contig) - { - rcv_noncontig_buff = MPL_malloc(recv_size*size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendbuf == MPI_IN_PLACE) - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - MPIR_Localcopy(recvbuf + (rank*recvcount*extent), recvcount, recvtype, - rcv_noncontig_buff + (rank*recv_size), recv_size,MPI_CHAR); - } - } - - /* Alltoall is much simpler if bytes are required because we don't need to - * malloc displ/count arrays and copy things - */ - - - pami_xfer_t alltoall; -#ifdef TRACE_ON - const pami_metadata_t *my_alltoall_md; - my_alltoall_md = &mpid->coll_metadata[PAMI_XFER_ALLTOALL][0][0]; - TRACE_ERR("Using alltoall protocol %s\n", my_alltoall_md->name); -#endif - alltoall.cb_done = cb_alltoall; - alltoall.cookie = (void *)&active; - alltoall.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLTOALL][0][0]; - alltoall.cmd.xfer_alltoall.stype = PAMI_TYPE_BYTE;/* stype is ignored when sndbuf == PAMI_IN_PLACE */ - alltoall.cmd.xfer_alltoall.stypecount = send_size; - alltoall.cmd.xfer_alltoall.sndbuf = sbuf; - - if(sendbuf == MPI_IN_PLACE) - { - alltoall.cmd.xfer_alltoall.sndbuf = PAMI_IN_PLACE; - } - alltoall.cmd.xfer_alltoall.rcvbuf = rbuf; - alltoall.cmd.xfer_alltoall.rtypecount = recv_size; - alltoall.cmd.xfer_alltoall.rtype = PAMI_TYPE_BYTE; - - MPIDI_Context_post(MPIDI_Context[0], &alltoall_post.state, - MPIDI_Pami_post_wrapper, (void *)&alltoall); - - TRACE_ERR("Waiting on active\n"); - MPID_PROGRESS_WAIT_WHILE(active); - - if(!rcv_contig) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size*size, MPI_CHAR, - recvbuf, recvcount*size, recvtype); - MPL_free(rcv_noncontig_buff); - } - if(!snd_contig) MPL_free(snd_noncontig_buff); - - TRACE_ERR("Leaving MPIDO_Alltoall_optimized\n"); - return MPI_SUCCESS; -} - - -int -MPIDO_CSWrapper_alltoall(pami_xfer_t *alltoall, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *sbuf; - MPIDI_coll_check_in_place(alltoall->cmd.xfer_alltoall.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( alltoall->cmd.xfer_alltoall.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( alltoall->cmd.xfer_alltoall.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Alltoall_intra(sbuf, - alltoall->cmd.xfer_alltoall.stypecount, sendtype, - alltoall->cmd.xfer_alltoall.rcvbuf, - alltoall->cmd.xfer_alltoall.rtypecount, recvtype, - comm_ptr, &mpierrno); - if(alltoall->cb_done && rc == 0) - alltoall->cb_done(NULL, alltoall->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/alltoall/mpido_ialltoall.c b/src/mpid/pamid/src/coll/alltoall/mpido_ialltoall.c deleted file mode 100644 index 7ecfb2c..0000000 --- a/src/mpid/pamid/src/coll/alltoall/mpido_ialltoall.c +++ /dev/null @@ -1,76 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/alltoall/mpido_ialltoall.c - * \brief ??? - */ - -/*#define TRACE_ON*/ - -#include <mpidimpl.h> - -int MPIDO_Ialltoall(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - TRACE_ERR("Entering MPIDO_Ialltoall\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Ialltoall_impl() function to invoke the mpich nbc implementation - * of MPI_Ialltoall(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ialltoall() with a blocking MPIR_Alltoall(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking alltoall algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Alltoall_impl(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - comm_ptr, &mpierrno); - - /* - * The blocking allitoall has completed - create and complete a - * MPIR_Request object so the MPIR_Ialltoall_impl() function does not - * perform an additional ialltoall. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - TRACE_ERR("Leaving MPIDO_Ialltoall\n"); - return 0; -} diff --git a/src/mpid/pamid/src/coll/alltoallv/Makefile.mk b/src/mpid/pamid/src/coll/alltoallv/Makefile.mk deleted file mode 100644 index 4e95a3c..0000000 --- a/src/mpid/pamid/src/coll/alltoallv/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/alltoallv/mpido_alltoallv.c \ - src/mpid/pamid/src/coll/alltoallv/mpido_ialltoallv.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/alltoallv/mpido_alltoallv.c b/src/mpid/pamid/src/coll/alltoallv/mpido_alltoallv.c deleted file mode 100644 index a66fdea..0000000 --- a/src/mpid/pamid/src/coll/alltoallv/mpido_alltoallv.c +++ /dev/null @@ -1,734 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/alltoallv/mpido_alltoallv.c - * \brief ??? - */ -/* #define TRACE_ON */ - -#include <mpidimpl.h> - -static void cb_alltoallv(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *)clientdata; - TRACE_ERR("alltoallv callback enter, active: %d\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - - -int MPIDO_Alltoallv(const void *sendbuf, - const int *sendcounts, - const int *senddispls, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *recvdispls, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Alltoallv\n"); - volatile unsigned active = 1; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - int recvok=PAMI_SUCCESS, sendok=PAMI_SUCCESS; - int sndtypelen, rcvtypelen, snd_contig=0, rcv_contig=0; - MPIDU_Datatype*sdt, *rdt; - pami_type_t stype, rtype; - MPI_Aint sdt_true_lb, rdt_true_lb; - MPIDI_Post_coll_t alltoallv_post; - int tmp; - const int rank = comm_ptr->rank; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_ALLTOALLV_INT]; - const int size = comm_ptr->local_size; - int sendcontinuous , recvcontinuous=0; - size_t recv_size=0, send_size=0; - size_t totalrecvcount=0; - int *lrecvdispls = NULL; /* possible local displs calculated for noncontinous */ - int *lsenddispls = NULL;/* possible local displs calculated for noncontinous */ - int *lrecvcounts = NULL;/* possible local counts calculated for noncontinous */ - int *lsendcounts = NULL;/* possible local counts calculated for noncontinous */ - const int *precvdispls = recvdispls; /* pointer to displs to use as pami parmi */ - const int *psenddispls = senddispls; /* pointer to displs to use as pami parmi */ - const int *precvcounts = recvcounts; /* pointer to counts to use as pami parmi */ - const int *psendcounts = sendcounts; /* pointer to counts to use as pami parmi */ - int inplace = sendbuf == MPI_IN_PLACE? 1 : 0; - if(selected_type == MPID_COLL_USE_MPICH) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH alltoallv algorithm\n"); - return MPIR_Alltoallv(sendbuf, sendcounts, senddispls, sendtype, - recvbuf, recvcounts, recvdispls, recvtype, - comm_ptr, mpierrno); - } - - if(!inplace) - { - sendok = MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp); - MPIDI_Datatype_get_info(1, sendtype, snd_contig, sndtypelen, sdt, sdt_true_lb); - sbuf = (char *)sendbuf + sdt_true_lb; - if(!snd_contig || (sendok != PAMI_SUCCESS)) - { - stype = PAMI_TYPE_UNSIGNED_CHAR; - size_t totalsendcount = sendcounts[0]; - sendcontinuous = senddispls[0] == 0? 1 : 0 ; - int i; - psenddispls = lsenddispls = MPL_malloc(size*sizeof(int)); - psendcounts = lsendcounts = MPL_malloc(size*sizeof(int)); - lsenddispls[0]= 0; - lsendcounts[0]= sndtypelen * sendcounts[0]; - for(i=1; i<size; ++i) - { - lsenddispls[i]= sndtypelen * totalsendcount; - totalsendcount += sendcounts[i]; - if(senddispls[i] != (senddispls[i-1] + (sendcounts[i-1]*sndtypelen))) - sendcontinuous = 0; - lsendcounts[i]= sndtypelen * sendcounts[i]; - } - send_size = sndtypelen * totalsendcount; - TRACE_ERR("Pack receive sndv_contig %zu, sendok %zd, totalsendcount %zu, sendcontinuous %zu, sndtypelen %zu, send_size %zu\n", - (size_t)snd_contig, (size_t)sendok, (size_t)totalsendcount, (size_t)sendcontinuous, (size_t) sndtypelen, (size_t)send_size); - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendcontinuous) - { - MPIR_Localcopy(sendbuf, totalsendcount, sendtype, - snd_noncontig_buff, send_size,MPI_CHAR); - } - else - { - size_t extent; - MPIDU_Datatype_get_extent_macro(sendtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)sendbuf + senddispls[i]*extent; - char* rcbuf = (char*)snd_noncontig_buff + psenddispls[i]; - MPIR_Localcopy(scbuf, sendcounts[i], sendtype, - rcbuf, psendcounts[i], MPI_CHAR); - TRACE_ERR("Pack send src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)senddispls[i],(size_t)i,(size_t)sendcounts[i],(size_t)senddispls[i], *(int*)scbuf); - TRACE_ERR("Pack send dest displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)i,(size_t)psenddispls[i],(size_t)i,(size_t)psendcounts[i],(size_t)psenddispls[i], *(int*)rcbuf); - } - } - } - } - MPIDI_Datatype_get_info(1, recvtype, rcv_contig, rcvtypelen, rdt, rdt_true_lb); - recvok = MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp); - rbuf = (char *)recvbuf + rdt_true_lb; - if(!rcv_contig || (recvok != PAMI_SUCCESS)) - { - rtype = PAMI_TYPE_UNSIGNED_CHAR; - totalrecvcount = recvcounts[0]; - recvcontinuous = recvdispls[0] == 0? 1 : 0 ; - int i; - precvdispls = lrecvdispls = MPL_malloc(size*sizeof(int)); - precvcounts = lrecvcounts = MPL_malloc(size*sizeof(int)); - lrecvdispls[0]= 0; - lrecvcounts[0]= rcvtypelen * recvcounts[0]; - for(i=1; i<size; ++i) - { - lrecvdispls[i]= rcvtypelen * totalrecvcount; - totalrecvcount += recvcounts[i]; - if(recvdispls[i] != (recvdispls[i-1] + (recvcounts[i-1]*rcvtypelen))) - recvcontinuous = 0; - lrecvcounts[i]= rcvtypelen * recvcounts[i]; - } - recv_size = rcvtypelen * totalrecvcount; - TRACE_ERR("Pack receive rcv_contig %zu, recvok %zd, totalrecvcount %zu, recvcontinuous %zu, rcvtypelen %zu, recv_size %zu\n", - (size_t)rcv_contig, (size_t)recvok, (size_t)totalrecvcount, (size_t)recvcontinuous,(size_t)rcvtypelen, (size_t)recv_size); - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - /* need to copy it now if it's used for the send buffer and then do not do in place */ - if(inplace) - { - inplace = 0; - stype = PAMI_TYPE_UNSIGNED_CHAR; - size_t totalsendcount = recvcounts[0]; - sendcontinuous = recvdispls[0] == 0? 1 : 0 ; - int i; - psenddispls = lsenddispls = MPL_malloc(size*sizeof(int)); - psendcounts = lsendcounts = MPL_malloc(size*sizeof(int)); - lsenddispls[0]= 0; - lsendcounts[0]= rcvtypelen * recvcounts[0]; - for(i=1; i<size; ++i) - { - lsenddispls[i]= rcvtypelen * totalsendcount; - totalsendcount += recvcounts[i]; - if(recvdispls[i] != (recvdispls[i-1] + (recvcounts[i-1]*rcvtypelen))) - sendcontinuous = 0; - lsendcounts[i]= rcvtypelen * recvcounts[i]; - } - send_size = rcvtypelen * totalsendcount; - TRACE_ERR("Pack MPI_IN_PLACE receive sndv_contig %zu, sendok %zd, totalsendcount %zu, sendcontinuous %zu, rcvtypelen %zu, send_size %zu\n", - (size_t)snd_contig, (size_t)sendok, (size_t)totalsendcount, (size_t)sendcontinuous, (size_t) rcvtypelen, (size_t)send_size); - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendcontinuous) - { - MPIR_Localcopy(recvbuf, totalsendcount, recvtype, - snd_noncontig_buff, send_size,MPI_CHAR); - } - else - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)recvbuf + recvdispls[i]*extent; - char* rcbuf = (char*)snd_noncontig_buff + psenddispls[i]; - MPIR_Localcopy(scbuf, recvcounts[i], recvtype, - rcbuf, psendcounts[i], MPI_CHAR); - TRACE_ERR("Pack send src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)recvdispls[i],(size_t)i,(size_t)recvcounts[i],(size_t)recvdispls[i], *(int*)scbuf); - TRACE_ERR("Pack send dest extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)psenddispls[i],(size_t)i,(size_t)psendcounts[i],(size_t)psenddispls[i], *(int*)rcbuf); - } - } - } - } - - - pami_xfer_t alltoallv; - pami_algorithm_t my_alltoallv; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized alltoallv was selected\n"); - my_alltoallv = mpid->opt_protocol[PAMI_XFER_ALLTOALLV_INT][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_ALLTOALLV_INT][0]; - queryreq = mpid->must_query[PAMI_XFER_ALLTOALLV_INT][0]; - } - else - { /* is this purely an else? or do i need to check for some other selectedvar... */ - TRACE_ERR("Alltoallv specified by user\n"); - my_alltoallv = mpid->user_selected[PAMI_XFER_ALLTOALLV_INT]; - my_md = &mpid->user_metadata[PAMI_XFER_ALLTOALLV_INT]; - queryreq = selected_type; - } - alltoallv.algorithm = my_alltoallv; - char *pname = my_md->name; - - - alltoallv.cb_done = cb_alltoallv; - alltoallv.cookie = (void *)&active; - /* We won't bother with alltoallv since MPI is always going to be ints. */ - if(inplace) - { - if(unlikely(verbose)) - fprintf(stderr,"alltoallv MPI_IN_PLACE buffering\n"); - alltoallv.cmd.xfer_alltoallv_int.stype = rtype; - alltoallv.cmd.xfer_alltoallv_int.sdispls = (int *) precvdispls; - alltoallv.cmd.xfer_alltoallv_int.stypecounts = (int *) precvcounts; - alltoallv.cmd.xfer_alltoallv_int.sndbuf = PAMI_IN_PLACE; - } - else - { - alltoallv.cmd.xfer_alltoallv_int.stype = stype; - alltoallv.cmd.xfer_alltoallv_int.sdispls = (int *) psenddispls; - alltoallv.cmd.xfer_alltoallv_int.stypecounts = (int *) psendcounts; - alltoallv.cmd.xfer_alltoallv_int.sndbuf = sbuf; - } - alltoallv.cmd.xfer_alltoallv_int.rcvbuf = rbuf; - - alltoallv.cmd.xfer_alltoallv_int.rdispls = (int *) precvdispls; - alltoallv.cmd.xfer_alltoallv_int.rtypecounts = (int *) precvcounts; - alltoallv.cmd.xfer_alltoallv_int.rtype = rtype; - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying alltoallv protocol %s, type was %d\n", pname, queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; -/* Can't check ranges like this. Non-local. Comment out for now. - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb; - MPIDU_Datatype*data_ptr; - int data_size, data_contig; - MPIDI_Datatype_get_info(??, sendtype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } -*/ - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&alltoallv); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH alltoallv\n", pname); - MPIDI_Update_last_algorithm(comm_ptr, "ALLTOALLV_MPICH"); - return MPIR_Alltoallv(sendbuf, sendcounts, senddispls, sendtype, - recvbuf, recvcounts, recvdispls, recvtype, - comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", pname); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for alltoallv on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - - MPIDI_Context_post(MPIDI_Context[0], &alltoallv_post.state, - MPIDI_Pami_post_wrapper, (void *)&alltoallv); - - TRACE_ERR("%d waiting on active %d\n", rank, active); - MPID_PROGRESS_WAIT_WHILE(active); - - if(!rcv_contig || (recvok != PAMI_SUCCESS)) - { - if(recvcontinuous) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size,MPI_CHAR, - recvbuf, totalrecvcount, recvtype); - } - else - { - size_t extent; - int i; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)rcv_noncontig_buff+ precvdispls[i]; - char* rcbuf = (char*)recvbuf + recvdispls[i]*extent; - MPIR_Localcopy(scbuf, precvcounts[i], MPI_CHAR, - rcbuf, recvcounts[i], recvtype); - TRACE_ERR("Pack recv src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)precvdispls[i],(size_t)i,(size_t)precvcounts[i],(size_t)precvdispls[i], *(int*)scbuf); - TRACE_ERR("Pack recv dest extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)recvdispls[i],(size_t)i,(size_t)recvcounts[i],(size_t)recvdispls[i], *(int*)rcbuf); - } - } - MPL_free(rcv_noncontig_buff); - } - if(!snd_contig || (sendok != PAMI_SUCCESS)) MPL_free(snd_noncontig_buff); - if(lrecvdispls) MPL_free(lrecvdispls); - if(lsenddispls) MPL_free(lsenddispls); - if(lrecvcounts) MPL_free(lrecvcounts); - if(lsendcounts) MPL_free(lsendcounts); - - TRACE_ERR("Leaving alltoallv\n"); - - - return 0; -} - - -int MPIDO_Alltoallv_simple(const void *sendbuf, - const int *sendcounts, - const int *senddispls, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *recvdispls, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Alltoallv_optimized\n"); - volatile unsigned active = 1; - int sndtypelen, rcvtypelen, snd_contig = 1, rcv_contig = 1; - MPIDU_Datatype*sdt, *rdt; - pami_type_t stype = NULL, rtype; - MPI_Aint sdt_true_lb = 0, rdt_true_lb; - MPIDI_Post_coll_t alltoallv_post; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - int recvok=PAMI_SUCCESS, sendok=PAMI_SUCCESS; - int tmp; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - int sendcontinuous , recvcontinuous=0; - size_t recv_size=0, send_size=0; - size_t totalrecvcount=0; - int *lrecvdispls = NULL; /* possible local displs calculated for noncontinous */ - int *lsenddispls = NULL;/* possible local displs calculated for noncontinous */ - int *lrecvcounts = NULL;/* possible local counts calculated for noncontinous */ - int *lsendcounts = NULL;/* possible local counts calculated for noncontinous */ - const int *precvdispls = recvdispls; /* pointer to displs to use as pami parmi */ - const int *psenddispls = senddispls; /* pointer to displs to use as pami parmi */ - const int *precvcounts = recvcounts; /* pointer to counts to use as pami parmi */ - const int *psendcounts = sendcounts; /* pointer to counts to use as pami parmi */ - int inplace = sendbuf == MPI_IN_PLACE? 1 : 0; - - - - - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - /* We don't pack and unpack in alltoallv as we do in alltoall because alltoallv has - more overhead of book keeping for sendcounts/displs and recvcounts/displs. Since, - alltoallv is non-rooted and all tasks has type info, decision to punt to mpich - will be the same on all tasks */ - - - /* Check if collsel has MPICH algorithm as the best performing one, if so, call MPICH now w/o doing any conversions */ - MPIDI_Datatype_get_info(1, recvtype, rcv_contig, rcvtypelen, rdt, rdt_true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_ALLTOALLV_INT, rcvtypelen * recvcounts[0], advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Alltoallv(sendbuf, sendcounts, senddispls, sendtype, - recvbuf, recvcounts, recvdispls, recvtype, - comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - /* Now do checks on send data and datatypes (contig and contin) and do necessary conversions */ - if(!inplace) - { - sendok = MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp); - MPIDI_Datatype_get_info(1, sendtype, snd_contig, sndtypelen, sdt, sdt_true_lb); - sbuf = (char *)sendbuf + sdt_true_lb; - if(!snd_contig || (sendok != PAMI_SUCCESS)) - { - stype = PAMI_TYPE_UNSIGNED_CHAR; - size_t totalsendcount = sendcounts[0]; - sendcontinuous = senddispls[0] == 0? 1 : 0 ; - int i; - psenddispls = lsenddispls = MPL_malloc(size*sizeof(int)); - psendcounts = lsendcounts = MPL_malloc(size*sizeof(int)); - lsenddispls[0]= 0; - lsendcounts[0]= sndtypelen * sendcounts[0]; - for(i=1; i<size; ++i) - { - lsenddispls[i]= sndtypelen * totalsendcount; - totalsendcount += sendcounts[i]; - if(senddispls[i] != (senddispls[i-1] + (sendcounts[i-1]))) - sendcontinuous = 0; - lsendcounts[i]= sndtypelen * sendcounts[i]; - } - send_size = sndtypelen * totalsendcount; - TRACE_ERR("Pack receive sndv_contig %zu, sendok %zd, totalsendcount %zu, sendcontinuous %zu, sndtypelen %zu, send_size %zu\n", - (size_t)snd_contig, (size_t)sendok, (size_t)totalsendcount, (size_t)sendcontinuous, (size_t) sndtypelen, (size_t)send_size); - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendcontinuous) - { - MPIR_Localcopy(sendbuf, totalsendcount, sendtype, - snd_noncontig_buff, send_size,MPI_CHAR); - } - else - { - size_t extent; - MPIDU_Datatype_get_extent_macro(sendtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)sendbuf + senddispls[i]*extent; - char* rcbuf = (char*)snd_noncontig_buff + psenddispls[i]; - MPIR_Localcopy(scbuf, sendcounts[i], sendtype, - rcbuf, psendcounts[i], MPI_CHAR); - TRACE_ERR("Pack send src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)senddispls[i],(size_t)i,(size_t)sendcounts[i],(size_t)senddispls[i], *(int*)scbuf); - TRACE_ERR("Pack send dest displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)i,(size_t)psenddispls[i],(size_t)i,(size_t)psendcounts[i],(size_t)psenddispls[i], *(int*)rcbuf); - } - } - } - } - - recvok = MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp); - rbuf = (char *)recvbuf + rdt_true_lb; - if(!rcv_contig || (recvok != PAMI_SUCCESS)) - { - rtype = PAMI_TYPE_UNSIGNED_CHAR; - totalrecvcount = recvcounts[0]; - recvcontinuous = recvdispls[0] == 0? 1 : 0 ; - int i; - precvdispls = lrecvdispls = MPL_malloc(size*sizeof(int)); - precvcounts = lrecvcounts = MPL_malloc(size*sizeof(int)); - lrecvdispls[0]= 0; - lrecvcounts[0]= rcvtypelen * recvcounts[0]; - for(i=1; i<size; ++i) - { - lrecvdispls[i]= rcvtypelen * totalrecvcount; - totalrecvcount += recvcounts[i]; - if(recvdispls[i] != (recvdispls[i-1] + (recvcounts[i-1]))) - recvcontinuous = 0; - lrecvcounts[i]= rcvtypelen * recvcounts[i]; - } - recv_size = rcvtypelen * totalrecvcount; - TRACE_ERR("Pack receive rcv_contig %zu, recvok %zd, totalrecvcount %zu, recvcontinuous %zu, rcvtypelen %zu, recv_size %zu\n", - (size_t)rcv_contig, (size_t)recvok, (size_t)totalrecvcount, (size_t)recvcontinuous,(size_t)rcvtypelen, (size_t)recv_size); - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - /* need to copy it now if it's used for the send buffer and then do not do in place */ - if(inplace) - { - inplace = 0; - stype = PAMI_TYPE_UNSIGNED_CHAR; - size_t totalsendcount = recvcounts[0]; - sendcontinuous = recvdispls[0] == 0? 1 : 0 ; - int i; - psenddispls = lsenddispls = MPL_malloc(size*sizeof(int)); - psendcounts = lsendcounts = MPL_malloc(size*sizeof(int)); - lsenddispls[0]= 0; - lsendcounts[0]= rcvtypelen * recvcounts[0]; - for(i=1; i<size; ++i) - { - lsenddispls[i]= rcvtypelen * totalsendcount; - totalsendcount += recvcounts[i]; - if(recvdispls[i] != (recvdispls[i-1] + (recvcounts[i-1]))) - sendcontinuous = 0; - lsendcounts[i]= rcvtypelen * recvcounts[i]; - } - send_size = rcvtypelen * totalsendcount; - TRACE_ERR("Pack MPI_IN_PLACE receive sndv_contig %zu, sendok %zd, totalsendcount %zu, sendcontinuous %zu, rcvtypelen %zu, send_size %zu\n", - (size_t)snd_contig, (size_t)sendok, (size_t)totalsendcount, (size_t)sendcontinuous, (size_t) rcvtypelen, (size_t)send_size); - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendcontinuous) - { - MPIR_Localcopy(recvbuf, totalsendcount, recvtype, - snd_noncontig_buff, send_size,MPI_CHAR); - } - else - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)recvbuf + recvdispls[i]*extent; - char* rcbuf = (char*)snd_noncontig_buff + psenddispls[i]; - MPIR_Localcopy(scbuf, recvcounts[i], recvtype, - rcbuf, psendcounts[i], MPI_CHAR); - TRACE_ERR("Pack send src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)recvdispls[i],(size_t)i,(size_t)recvcounts[i],(size_t)recvdispls[i], *(int*)scbuf); - TRACE_ERR("Pack send dest extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)psenddispls[i],(size_t)i,(size_t)psendcounts[i],(size_t)psenddispls[i], *(int*)rcbuf); - } - } - } - } - - - - pami_xfer_t alltoallv; - alltoallv.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLTOALLV_INT][0][0]; - - alltoallv.cb_done = cb_alltoallv; - alltoallv.cookie = (void *)&active; - if(inplace) - { - alltoallv.cmd.xfer_alltoallv_int.stype = rtype; - alltoallv.cmd.xfer_alltoallv_int.sdispls = (int *) precvdispls; - alltoallv.cmd.xfer_alltoallv_int.stypecounts = (int *) precvcounts; - alltoallv.cmd.xfer_alltoallv_int.sndbuf = PAMI_IN_PLACE; - } - else - { - alltoallv.cmd.xfer_alltoallv_int.stype = stype; - alltoallv.cmd.xfer_alltoallv_int.sdispls = (int *) psenddispls; - alltoallv.cmd.xfer_alltoallv_int.stypecounts = (int *) psendcounts; - alltoallv.cmd.xfer_alltoallv_int.sndbuf = sbuf; - } - alltoallv.cmd.xfer_alltoallv_int.rcvbuf = rbuf; - - alltoallv.cmd.xfer_alltoallv_int.rdispls = (int *) precvdispls; - alltoallv.cmd.xfer_alltoallv_int.rtypecounts = (int *) precvcounts; - alltoallv.cmd.xfer_alltoallv_int.rtype = rtype; - - - MPIDI_Context_post(MPIDI_Context[0], &alltoallv_post.state, - MPIDI_Pami_post_wrapper, (void *)&alltoallv); - - TRACE_ERR("%d waiting on active %d\n", rank, active); - MPID_PROGRESS_WAIT_WHILE(active); - - if(!rcv_contig || (recvok != PAMI_SUCCESS)) - { - if(recvcontinuous) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size,MPI_CHAR, - recvbuf, totalrecvcount, recvtype); - } - else - { - size_t extent; - int i; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)rcv_noncontig_buff+ precvdispls[i]; - char* rcbuf = (char*)recvbuf + recvdispls[i]*extent; - MPIR_Localcopy(scbuf, precvcounts[i], MPI_CHAR, - rcbuf, recvcounts[i], recvtype); - TRACE_ERR("Pack recv src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)precvdispls[i],(size_t)i,(size_t)precvcounts[i],(size_t)precvdispls[i], *(int*)scbuf); - TRACE_ERR("Pack recv dest extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)recvdispls[i],(size_t)i,(size_t)recvcounts[i],(size_t)recvdispls[i], *(int*)rcbuf); - } - } - MPL_free(rcv_noncontig_buff); - } - if(!snd_contig || (sendok != PAMI_SUCCESS)) MPL_free(snd_noncontig_buff); - if(lrecvdispls) MPL_free(lrecvdispls); - if(lsenddispls) MPL_free(lsenddispls); - if(lrecvcounts) MPL_free(lrecvcounts); - if(lsendcounts) MPL_free(lsendcounts); - - - TRACE_ERR("Leaving alltoallv\n"); - - - return MPI_SUCCESS; -} - -int -MPIDO_CSWrapper_alltoallv(pami_xfer_t *alltoallv, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *sbuf; - MPIDI_coll_check_in_place(alltoallv->cmd.xfer_alltoallv_int.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( alltoallv->cmd.xfer_alltoallv_int.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( alltoallv->cmd.xfer_alltoallv_int.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Alltoallv(sbuf, - alltoallv->cmd.xfer_alltoallv_int.stypecounts, - alltoallv->cmd.xfer_alltoallv_int.sdispls, sendtype, - alltoallv->cmd.xfer_alltoallv_int.rcvbuf, - alltoallv->cmd.xfer_alltoallv_int.rtypecounts, - alltoallv->cmd.xfer_alltoallv_int.rdispls, recvtype, - comm_ptr, &mpierrno); - if(alltoallv->cb_done && rc == 0) - alltoallv->cb_done(NULL, alltoallv->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/alltoallv/mpido_ialltoallv.c b/src/mpid/pamid/src/coll/alltoallv/mpido_ialltoallv.c deleted file mode 100644 index a296664..0000000 --- a/src/mpid/pamid/src/coll/alltoallv/mpido_ialltoallv.c +++ /dev/null @@ -1,79 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/alltoallv/mpido_alltoallv.c - * \brief ??? - */ -/*#define TRACE_ON*/ - -#include <mpidimpl.h> - -int MPIDO_Ialltoallv(const void *sendbuf, - const int *sendcounts, - const int *senddispls, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *recvdispls, - MPI_Datatype recvtype, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - if(comm_ptr->rank == 0) - TRACE_ERR("Entering MPIDO_Ialltoallv\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Ialltoalliv_impl() function to invoke the mpich nbc implementation - * of MPI_Ialltoallv(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ialltoallv() with a blocking MPIR_Alltoallv(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking alltoallv algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Alltoallv_impl(sendbuf, sendcounts, senddispls, sendtype, - recvbuf, recvcounts, recvdispls, recvtype, - comm_ptr, &mpierrno); - - /* - * The blocking alltoallv has completed - create and complete a - * MPIR_Request object so the MPIR_Ialltoallv_impl() function does not - * perform an additional ialltoallv. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - - TRACE_ERR("Leaving ialltoallv\n"); - return 0; -} diff --git a/src/mpid/pamid/src/coll/alltoallw/Makefile.mk b/src/mpid/pamid/src/coll/alltoallw/Makefile.mk deleted file mode 100644 index 4393952..0000000 --- a/src/mpid/pamid/src/coll/alltoallw/Makefile.mk +++ /dev/null @@ -1,28 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/alltoallw/mpido_ialltoallw.c - - -endif BUILD_PAMID diff --git a/src/mpid/pamid/src/coll/alltoallw/mpido_ialltoallw.c b/src/mpid/pamid/src/coll/alltoallw/mpido_ialltoallw.c deleted file mode 100644 index 2569bfa..0000000 --- a/src/mpid/pamid/src/coll/alltoallw/mpido_ialltoallw.c +++ /dev/null @@ -1,79 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/alltoallv/mpido_alltoallw.c - * \brief ??? - */ -/*#define TRACE_ON*/ - -#include <mpidimpl.h> - -int MPIDO_Ialltoallw(const void *sendbuf, - const int *sendcounts, - const int *senddispls, - const MPI_Datatype *sendtypes, - void *recvbuf, - const int *recvcounts, - const int *recvdispls, - const MPI_Datatype *recvtypes, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - if(comm_ptr->rank == 0) - TRACE_ERR("Entering MPIDO_Ialltoallw\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Ialltoallw_impl() function to invoke the mpich nbc implementation - * of MPI_Ialltoallw(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ialltoallw() with a blocking MPIR_Alltoallw(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking alltoallw algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Alltoallw_impl(sendbuf, sendcounts, senddispls, sendtypes, - recvbuf, recvcounts, recvdispls, recvtypes, - comm_ptr, &mpierrno); - - /* - * The blocking alltoallw has completed - create and complete a - * MPIR_Request object so the MPIR_Ialltoallw_impl() function does not - * perform an additional ialltoallw. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - - TRACE_ERR("Leaving ialltoallw\n"); - return 0; -} diff --git a/src/mpid/pamid/src/coll/barrier/Makefile.mk b/src/mpid/pamid/src/coll/barrier/Makefile.mk deleted file mode 100644 index 3d49921..0000000 --- a/src/mpid/pamid/src/coll/barrier/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/barrier/mpido_barrier.c \ - src/mpid/pamid/src/coll/barrier/mpido_ibarrier.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/barrier/mpido_barrier.c b/src/mpid/pamid/src/coll/barrier/mpido_barrier.c deleted file mode 100644 index 9e91334..0000000 --- a/src/mpid/pamid/src/coll/barrier/mpido_barrier.c +++ /dev/null @@ -1,146 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/barrier/mpido_barrier.c - * \brief ??? - */ - -/* #define TRACE_ON */ - -#include <mpidimpl.h> - -static void cb_barrier(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *) clientdata; - TRACE_ERR("callback. enter: %d\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - -int MPIDO_Barrier(MPIR_Comm *comm_ptr, int *mpierrno) -{ - TRACE_ERR("Entering MPIDO_Barrier\n"); - volatile unsigned active=1; - MPIDI_Post_coll_t barrier_post; - pami_xfer_t barrier; - pami_algorithm_t my_barrier; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_BARRIER]; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (comm_ptr->rank == 0); -#endif - - if(unlikely(selected_type == MPID_COLL_USE_MPICH)) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH barrier\n"); - TRACE_ERR("Using MPICH Barrier\n"); - return MPIR_Barrier(comm_ptr, mpierrno); - } - - barrier.cb_done = cb_barrier; - barrier.cookie = (void *)&active; - if(likely(selected_type == MPID_COLL_OPTIMIZED)) - { - TRACE_ERR("Optimized barrier (%s) was pre-selected\n", mpid->opt_protocol_md[PAMI_XFER_BARRIER][0].name); - my_barrier = mpid->opt_protocol[PAMI_XFER_BARRIER][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_BARRIER][0]; - queryreq = mpid->must_query[PAMI_XFER_BARRIER][0]; - } - else - { - TRACE_ERR("Barrier (%s) was specified by user\n", mpid->user_metadata[PAMI_XFER_BARRIER].name); - my_barrier = mpid->user_selected[PAMI_XFER_BARRIER]; - my_md = &mpid->user_metadata[PAMI_XFER_BARRIER]; - queryreq = selected_type; - } - - barrier.algorithm = my_barrier; - /* There is no support for query-required barrier protocols here */ - MPID_assert(queryreq != MPID_COLL_ALWAYS_QUERY); - MPID_assert(queryreq != MPID_COLL_CHECK_FN_REQUIRED); - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for barrier on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - MPIDI_Context_post(MPIDI_Context[0], &barrier_post.state, - MPIDI_Pami_post_wrapper, (void *)&barrier); - - TRACE_ERR("advance spinning\n"); - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - MPID_PROGRESS_WAIT_WHILE(active); - TRACE_ERR("exiting mpido_barrier\n"); - return 0; -} - - -int MPIDO_Barrier_simple(MPIR_Comm *comm_ptr, int *mpierrno) -{ - TRACE_ERR("Entering MPIDO_Barrier_optimized\n"); - volatile unsigned active=1; - MPIDI_Post_coll_t barrier_post; - pami_xfer_t barrier; - pami_algorithm_t my_barrier; - const pami_metadata_t *my_barrier_md; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - - barrier.cb_done = cb_barrier; - barrier.cookie = (void *)&active; - my_barrier = mpid->coll_algorithm[PAMI_XFER_BARRIER][0][0]; - my_barrier_md = &mpid->coll_metadata[PAMI_XFER_BARRIER][0][0]; - barrier.algorithm = my_barrier; - - - TRACE_ERR("%s barrier\n", MPIDI_Process.context_post.active>0?"posting":"invoking"); - MPIDI_Context_post(MPIDI_Context[0], &barrier_post.state, - MPIDI_Pami_post_wrapper, (void *)&barrier); - TRACE_ERR("barrier %s rc: %d\n", MPIDI_Process.context_post.active>0?"posted":"invoked", rc); - - TRACE_ERR("advance spinning\n"); - MPIDI_Update_last_algorithm(comm_ptr, my_barrier_md->name); - MPID_PROGRESS_WAIT_WHILE(active); - TRACE_ERR("Exiting MPIDO_Barrier_optimized\n"); - return 0; -} - -int -MPIDO_CSWrapper_barrier(pami_xfer_t *barrier, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - int rc = MPIR_Barrier(comm_ptr, &mpierrno); - if(barrier->cb_done && rc == 0) - barrier->cb_done(NULL, barrier->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/barrier/mpido_ibarrier.c b/src/mpid/pamid/src/coll/barrier/mpido_ibarrier.c deleted file mode 100644 index eb093fa..0000000 --- a/src/mpid/pamid/src/coll/barrier/mpido_ibarrier.c +++ /dev/null @@ -1,62 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/barrier/mpido_ibarrier.c - * \brief ??? - */ - -/*#define TRACE_ON*/ - -#include <mpidimpl.h> - -static void cb_ibarrier(void *ctxt, void *clientdata, pami_result_t err) -{ - MPIR_Request *mpid_request = (MPIR_Request *) clientdata; - MPIDI_Request_complete_norelease_inline(mpid_request); -} - -int MPIDO_Ibarrier(MPIR_Comm *comm_ptr, MPIR_Request **request) -{ - TRACE_ERR("Entering MPIDO_Ibarrier\n"); - - /* - * There is actually no current pami optimization for this - * so just kick it back to MPICH if mpir_nbc is set, otherwise - * call the blocking MPIR_Barrier(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ibarrier() with a blocking MPIR_Barrier(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH barrier\n"); - TRACE_ERR("Using MPICH Barrier\n"); - - int mpierrno = 0; - int rc = MPIR_Barrier(comm_ptr, &mpierrno); - - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; -} diff --git a/src/mpid/pamid/src/coll/bcast/Makefile.mk b/src/mpid/pamid/src/coll/bcast/Makefile.mk deleted file mode 100644 index 3988631..0000000 --- a/src/mpid/pamid/src/coll/bcast/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/bcast/mpido_bcast.c \ - src/mpid/pamid/src/coll/bcast/mpido_ibcast.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/bcast/mpido_bcast.c b/src/mpid/pamid/src/coll/bcast/mpido_bcast.c deleted file mode 100644 index d10a916..0000000 --- a/src/mpid/pamid/src/coll/bcast/mpido_bcast.c +++ /dev/null @@ -1,394 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/bcast/mpido_bcast.c - * \brief ??? - */ - -/* #define TRACE_ON */ - -#include <mpidimpl.h> - -static void cb_bcast(void *ctxt, void *clientdata, pami_result_t err) -{ - int *active = (int *) clientdata; - TRACE_ERR("callback enter, active: %d\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - -int MPIDO_Bcast(void *buffer, - int count, - MPI_Datatype datatype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ - TRACE_ERR("in mpido_bcast\n"); - const size_t BCAST_LIMIT = 0x40000000; - int data_contig, rc; - void *data_buffer = NULL, - *noncontig_buff = NULL; - volatile unsigned active = 1; - MPI_Aint data_true_lb = 0; - MPIDU_Datatype*data_ptr; - MPIDU_Segment segment; - MPIDI_Post_coll_t bcast_post; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int rank = comm_ptr->rank; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const int selected_type = mpid->user_selected_type[PAMI_XFER_BROADCAST]; - - /* Must calculate data_size based on count=1 in case it's total size is > integer */ - int data_size_one; - MPIDI_Datatype_get_info(1, datatype, - data_contig, data_size_one, data_ptr, data_true_lb); - /* do this calculation once and use twice */ - const size_t data_size_sz = (size_t)data_size_one*(size_t)count; - if(unlikely(verbose)) - fprintf(stderr,"bcast count %d, size %d (%#zX), root %d, buffer %p\n", - count,data_size_one, (size_t)data_size_one*(size_t)count, root,buffer); - if(unlikely( data_size_sz > BCAST_LIMIT) ) - { - void *new_buffer=buffer; - int c, new_count = (int)BCAST_LIMIT/data_size_one; - MPID_assert(new_count > 0); - - for(c=1; ((size_t)c*(size_t)new_count) <= (size_t)count; ++c) - { - if ((rc = MPIDO_Bcast(new_buffer, - new_count, - datatype, - root, - comm_ptr, - mpierrno)) != MPI_SUCCESS) - return rc; - new_buffer = (char*)new_buffer + (size_t)data_size_one*(size_t)new_count; - } - new_count = count % new_count; /* 0 is ok, just returns no-op */ - return MPIDO_Bcast(new_buffer, - new_count, - datatype, - root, - comm_ptr, - mpierrno); - } - - /* Must use data_size based on count for byte bcast processing. - Previously calculated as a size_t but large data_sizes were - handled above so this cast to int should be fine here. - */ - const int data_size = (int)data_size_sz; - - if(selected_type == MPID_COLL_USE_MPICH || data_size == 0) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH bcast algorithm\n"); - MPIDI_Update_last_algorithm(comm_ptr,"BCAST_MPICH"); - return MPIR_Bcast_intra(buffer, count, datatype, root, comm_ptr, mpierrno); - } - - data_buffer = (char *)buffer + data_true_lb; - - if(!data_contig) - { - noncontig_buff = MPL_malloc(data_size); - data_buffer = noncontig_buff; - if(noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(rank == root) - { - DLOOP_Offset last = data_size; - MPIDU_Segment_init(buffer, count, datatype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, noncontig_buff); - } - } - - pami_xfer_t bcast; - pami_algorithm_t my_bcast; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - - bcast.cb_done = cb_bcast; - bcast.cookie = (void *)&active; - bcast.cmd.xfer_broadcast.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - bcast.algorithm = mpid->user_selected[PAMI_XFER_BROADCAST]; - bcast.cmd.xfer_broadcast.buf = data_buffer; - bcast.cmd.xfer_broadcast.type = PAMI_TYPE_BYTE; - /* Needs to be sizeof(type)*count since we are using bytes as * the generic type */ - bcast.cmd.xfer_broadcast.typecount = data_size; - - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized bcast (%s) and (%s) were pre-selected\n", - mpid->opt_protocol_md[PAMI_XFER_BROADCAST][0].name, - mpid->opt_protocol_md[PAMI_XFER_BROADCAST][1].name); - - if(mpid->cutoff_size[PAMI_XFER_BROADCAST][1] != 0)/* SSS: There is FCA cutoff (FCA only sets cutoff for [PAMI_XFER_BROADCAST][1]) */ - { - if(data_size <= mpid->cutoff_size[PAMI_XFER_BROADCAST][1]) - { - my_bcast = mpid->opt_protocol[PAMI_XFER_BROADCAST][1]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_BROADCAST][1]; - queryreq = mpid->must_query[PAMI_XFER_BROADCAST][1]; - } - else - { - return MPIR_Bcast_intra(buffer, count, datatype, root, comm_ptr, mpierrno); - } - } - - if(data_size > mpid->cutoff_size[PAMI_XFER_BROADCAST][0]) - { - my_bcast = mpid->opt_protocol[PAMI_XFER_BROADCAST][1]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_BROADCAST][1]; - queryreq = mpid->must_query[PAMI_XFER_BROADCAST][1]; - } - else - { - my_bcast = mpid->opt_protocol[PAMI_XFER_BROADCAST][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_BROADCAST][0]; - queryreq = mpid->must_query[PAMI_XFER_BROADCAST][0]; - } - } - else - { - TRACE_ERR("Bcast (%s) was specified by user\n", - mpid->user_metadata[PAMI_XFER_BROADCAST].name); - my_bcast = mpid->user_selected[PAMI_XFER_BROADCAST]; - my_md = &mpid->user_metadata[PAMI_XFER_BROADCAST]; - queryreq = selected_type; - } - - bcast.algorithm = my_bcast; - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying bcast protocol %s, type was: %d\n", - my_md->name, queryreq); - if(my_md->check_fn != NULL) /* calling the check fn is sufficient */ - { - result = my_md->check_fn(&bcast); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - } - else /* no check_fn, manually look at the metadata fields */ - { - TRACE_ERR("Optimzed selection line %d\n",__LINE__); - /* Check if the message range if restricted */ - if(my_md->check_correct.values.rangeminmax) - { - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - /* \todo check the rest of the metadata */ - } - TRACE_ERR("bitmask: %#X\n", result.bitmask); - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH bcast algorithm - query fn failed\n"); - MPIDI_Update_last_algorithm(comm_ptr,"BCAST_MPICH"); - return MPIR_Bcast_intra(buffer, count, datatype, root, comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for bcast on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - - MPIDI_Context_post(MPIDI_Context[0], &bcast_post.state, MPIDI_Pami_post_wrapper, (void *)&bcast); - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - MPID_PROGRESS_WAIT_WHILE(active); - TRACE_ERR("bcast done\n"); - - if(!data_contig) - { - if(rank != root) - MPIR_Localcopy(noncontig_buff, data_size, MPI_CHAR, - buffer, count, datatype); - MPL_free(noncontig_buff); - } - - TRACE_ERR("leaving bcast\n"); - return 0; -} - - -int MPIDO_Bcast_simple(void *buffer, - int count, - MPI_Datatype datatype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ - TRACE_ERR("Entering MPIDO_Bcast_optimized\n"); - - int data_contig; - void *data_buffer = NULL, - *noncontig_buff = NULL; - volatile unsigned active = 1; - MPI_Aint data_true_lb = 0; - MPIDU_Datatype*data_ptr; - MPIDU_Segment segment; - MPIDI_Post_coll_t bcast_post; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int rank = comm_ptr->rank; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - /* Must calculate data_size based on count=1 in case it's total size is > integer */ - int data_size_one; - MPIDI_Datatype_get_info(1, datatype, - data_contig, data_size_one, data_ptr, data_true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_BROADCAST, data_size_one * count, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Bcast_intra(buffer, count, datatype, root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - const int data_size = data_size_one*(size_t)count; - - data_buffer = (char *)buffer + data_true_lb; - - if(!data_contig) - { - noncontig_buff = MPL_malloc(data_size); - data_buffer = noncontig_buff; - if(noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(rank == root) - { - DLOOP_Offset last = data_size; - MPIDU_Segment_init(buffer, count, datatype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, noncontig_buff); - } - } - - pami_xfer_t bcast; - const pami_metadata_t *my_bcast_md; - - bcast.cb_done = cb_bcast; - bcast.cookie = (void *)&active; - bcast.cmd.xfer_broadcast.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - bcast.algorithm = mpid->coll_algorithm[PAMI_XFER_BROADCAST][0][0]; - bcast.cmd.xfer_broadcast.buf = data_buffer; - bcast.cmd.xfer_broadcast.type = PAMI_TYPE_BYTE; - /* Needs to be sizeof(type)*count since we are using bytes as * the generic type */ - bcast.cmd.xfer_broadcast.typecount = data_size; - my_bcast_md = &mpid->coll_metadata[PAMI_XFER_BROADCAST][0][0]; - - MPIDI_Context_post(MPIDI_Context[0], &bcast_post.state, MPIDI_Pami_post_wrapper, (void *)&bcast); - MPIDI_Update_last_algorithm(comm_ptr, my_bcast_md->name); - MPID_PROGRESS_WAIT_WHILE(active); - TRACE_ERR("bcast done\n"); - - if(!data_contig) - { - if(rank != root) - MPIR_Localcopy(noncontig_buff, data_size, MPI_CHAR, - buffer, count, datatype); - MPL_free(noncontig_buff); - } - - TRACE_ERR("Exiting MPIDO_Bcast_optimized\n"); - return 0; -} - - -int -MPIDO_CSWrapper_bcast(pami_xfer_t *bcast, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype type; - int rc = MPIDI_Dtpami_to_dtmpi( bcast->cmd.xfer_broadcast.type, - &type, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Bcast_intra(bcast->cmd.xfer_broadcast.buf, - bcast->cmd.xfer_broadcast.typecount, type, - bcast->cmd.xfer_broadcast.root, comm_ptr, &mpierrno); - if(bcast->cb_done && rc == 0) - bcast->cb_done(NULL, bcast->cookie, PAMI_SUCCESS); - return rc; -} - diff --git a/src/mpid/pamid/src/coll/bcast/mpido_ibcast.c b/src/mpid/pamid/src/coll/bcast/mpido_ibcast.c deleted file mode 100644 index 85134dd..0000000 --- a/src/mpid/pamid/src/coll/bcast/mpido_ibcast.c +++ /dev/null @@ -1,67 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/bcast/mpido_ibcast.c - * \brief ??? - */ - -/*#define TRACE_ON*/ - -#include <mpidimpl.h> - -int MPIDO_Ibcast(void *buffer, - int count, - MPI_Datatype datatype, - int root, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - TRACE_ERR("in mpido_ibcast\n"); - - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to MPIR_Ibcast_impl - * to invoke the mpich nbc implementation of MPI_Ibcast(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ibcast() with a blocking MPIR_Bcast(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH bcast algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Bcast_intra(buffer, count, datatype, root, comm_ptr, &mpierrno); - - /* - * The blocking bcast has completed - create and complete a MPIR_Request - * object so the MPIR_Ibcast_impl() function does not perform the bcast. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - - TRACE_ERR("leaving ibcast\n"); - return 0; -} diff --git a/src/mpid/pamid/src/coll/coll_utils.c b/src/mpid/pamid/src/coll/coll_utils.c deleted file mode 100644 index 50a3d55..0000000 --- a/src/mpid/pamid/src/coll/coll_utils.c +++ /dev/null @@ -1,405 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/coll_utils.c - * \brief ??? - */ - -//#define TRACE_ON - -#include <mpidimpl.h> - -pami_result_t MPIDI_Pami_post_wrapper(pami_context_t context, void *cookie) -{ - TRACE_ERR("In post wrapper\n"); - TRACE_ERR("About to call collecive\n"); - PAMI_Collective(context, (pami_xfer_t *)cookie); - TRACE_ERR("Done calling collective\n"); - return PAMI_SUCCESS; -} - -/* some useful macros to make the comparisons less icky, esp given the */ -/* explosion of datatypes in MPI2.2 */ -#define isS_INT(x) ( (x)==MPI_INTEGER || \ - (x) == MPI_INT32_T || (x) == MPI_INTEGER4 || \ - (x) == MPI_INT) - -#define isUS_INT(x) ( (x) == MPI_UINT32_T || (x) == MPI_UNSIGNED) - -#define isS_SHORT(x) ( (x) == MPI_SHORT || (x) == MPI_INT16_T || \ - (x) == MPI_INTEGER2) - -#define isUS_SHORT(x) ( (x) == MPI_UNSIGNED_SHORT || (x) == MPI_UINT16_T) - -#define isS_CHAR(x) ( (x) == MPI_SIGNED_CHAR || (x) == MPI_INT8_T || \ - (x) == MPI_INTEGER1 || (x) == MPI_CHAR) - -#define isUS_CHAR(x) ( (x) == MPI_BYTE || \ - (x) == MPI_UNSIGNED_CHAR || (x) == MPI_UINT8_T ) - -/* sizeof(longlong) == sizeof(long) == sizeof(uint64) on bgq */ -#define isS_LONG(x) ( (x) == MPI_LONG || (x) == MPI_AINT ) - -#define isUS_LONG(x) ( (x) == MPI_UNSIGNED_LONG ) - -#define isS_LONG_LONG(x) ( (x) == MPI_INT64_T || (x) == MPI_OFFSET || \ - (x) == MPI_INTEGER8 || (x) == MPI_LONG_LONG || \ - (x) == MPI_LONG_LONG_INT ) - -#define isUS_LONG_LONG(x) ( (x) == MPI_UINT64_T || (x) == MPI_UNSIGNED_LONG_LONG ) - - - -#define isFLOAT(x) ( (x) == MPI_FLOAT || (x) == MPI_REAL) - -#define isDOUBLE(x) ( (x) == MPI_DOUBLE || (x) == MPI_DOUBLE_PRECISION) - -#define isLONG_DOUBLE(x) ( (x) == MPI_LONG_DOUBLE ) - -#define isLOC_TYPE(x) ( (x) == MPI_2REAL || (x) == MPI_2DOUBLE_PRECISION || \ - (x) == MPI_2INTEGER || (x) == MPI_FLOAT_INT || \ - (x) == MPI_DOUBLE_INT || (x) == MPI_LONG_INT || \ - (x) == MPI_2INT || (x) == MPI_SHORT_INT || \ - (x) == MPI_LONG_DOUBLE_INT ) - -#define isPAMI_LOC_TYPE(x) ( (x) == PAMI_TYPE_LOC_2INT || (x) == PAMI_TYPE_LOC_FLOAT_INT || \ - (x) == PAMI_TYPE_LOC_DOUBLE_INT || (x) == PAMI_TYPE_LOC_SHORT_INT || \ - (x) == PAMI_TYPE_LOC_LONG_INT || (x) == PAMI_TYPE_LOC_LONGDOUBLE_INT || \ - (x) == PAMI_TYPE_LOC_2FLOAT || (x) == PAMI_TYPE_LOC_2DOUBLE ) - -#define isBOOL(x) ( (x) == MPI_C_BOOL ) - -#define isLOGICAL(x) ( (x) == MPI_LOGICAL ) - -#define isSINGLE_COMPLEX(x) ( (x) == MPI_COMPLEX || (x) == MPI_C_FLOAT_COMPLEX) - -#define isDOUBLE_COMPLEX(x) ( (x) == MPI_DOUBLE_COMPLEX || (x) == MPI_COMPLEX8 ||\ - (x) == MPI_C_DOUBLE_COMPLEX) - -/* known missing types: MPI_C_LONG_DOUBLE_COMPLEX */ - - -#define MUSUPPORTED 1 -#define MUUNSUPPORTED 0 - -/* for easier debug */ -void MPIDI_Op_to_string(MPI_Op op, char *string) -{ - switch(op) - { - case MPI_SUM: strcpy(string, "MPI_SUM"); break; - case MPI_PROD: strcpy(string, "MPI_PROD"); break; - case MPI_LAND: strcpy(string, "MPI_LAND"); break; - case MPI_LOR: strcpy(string, "MPI_LOR"); break; - case MPI_LXOR: strcpy(string, "MPI_LXOR"); break; - case MPI_BAND: strcpy(string, "MPI_BAND"); break; - case MPI_BOR: strcpy(string, "MPI_BOR"); break; - case MPI_BXOR: strcpy(string, "MPI_BXOR"); break; - case MPI_MAX: strcpy(string, "MPI_MAX"); break; - case MPI_MIN: strcpy(string, "MPI_MIN"); break; - case MPI_MINLOC: strcpy(string, "MPI_MINLOC"); break; - case MPI_MAXLOC: strcpy(string, "MPI_MAXLOC"); break; - default: strcpy(string, "Other"); break; - } -} - - -int MPIDI_Datatype_to_pami(MPI_Datatype dt, - pami_type_t *pdt, - MPI_Op op, - pami_data_function *pop, - int *musupport) -{ - *musupport = MUSUPPORTED; - *pdt = PAMI_TYPE_NULL; - if(op != -1) - *pop = PAMI_DATA_NOOP; - if(isS_INT(dt)) *pdt = PAMI_TYPE_SIGNED_INT; - else if(isUS_INT(dt)) *pdt = PAMI_TYPE_UNSIGNED_INT; - else if(isFLOAT(dt)) *pdt = PAMI_TYPE_FLOAT; - else if(isDOUBLE(dt)) *pdt = PAMI_TYPE_DOUBLE; - else if(isLONG_DOUBLE(dt)) *pdt = PAMI_TYPE_LONG_DOUBLE; - else - { - *musupport = MUUNSUPPORTED; - if(isS_CHAR(dt)) *pdt = PAMI_TYPE_SIGNED_CHAR; - else if(isUS_CHAR(dt)) *pdt = PAMI_TYPE_UNSIGNED_CHAR; - else if(isS_SHORT(dt)) *pdt = PAMI_TYPE_SIGNED_SHORT; - else if(isUS_SHORT(dt)) *pdt = PAMI_TYPE_UNSIGNED_SHORT; - else if(isS_LONG(dt)) *pdt = PAMI_TYPE_SIGNED_LONG; - else if(isUS_LONG(dt)) *pdt = PAMI_TYPE_UNSIGNED_LONG; - else if(isS_LONG_LONG(dt)) *pdt = PAMI_TYPE_SIGNED_LONG_LONG; - else if(isUS_LONG_LONG(dt)) *pdt = PAMI_TYPE_UNSIGNED_LONG_LONG; - else if(isSINGLE_COMPLEX(dt)) *pdt = PAMI_TYPE_SINGLE_COMPLEX; - else if(isDOUBLE_COMPLEX(dt)) *pdt = PAMI_TYPE_DOUBLE_COMPLEX; - else if(isLOC_TYPE(dt)) - { - switch(dt) - { - case MPI_2INT: *pdt = PAMI_TYPE_LOC_2INT; break; - case MPI_FLOAT_INT: *pdt = PAMI_TYPE_LOC_FLOAT_INT; break; - case MPI_DOUBLE_INT: *pdt = PAMI_TYPE_LOC_DOUBLE_INT; break; - case MPI_SHORT_INT: *pdt = PAMI_TYPE_LOC_SHORT_INT; break; - - case MPI_LONG_INT: *pdt = PAMI_TYPE_LOC_LONG_INT; break; - case MPI_LONG_DOUBLE_INT: *pdt = PAMI_TYPE_LOC_LONGDOUBLE_INT; break; - default: - /* The following is needed to catch missing fortran bindings */ - if (dt == MPI_2REAL) - *pdt = PAMI_TYPE_LOC_2FLOAT; - else if (dt == MPI_2DOUBLE_PRECISION) - *pdt = PAMI_TYPE_LOC_2DOUBLE; - else if (dt == MPI_2INTEGER) - *pdt = PAMI_TYPE_LOC_2INT; - } - /* - * There are some 2-element types that PAMI doesn't support, so we - * need to bail on anything that's left of the LOC_TYPEs - */ - if(*pdt == PAMI_TYPE_NULL) return -1; - - if(op == -1) return MPI_SUCCESS; - if(op == MPI_MINLOC) - { - *pop = PAMI_DATA_MINLOC; - return MPI_SUCCESS; - } - if(op == MPI_MAXLOC) - { - *pop = PAMI_DATA_MAXLOC; - return MPI_SUCCESS; - } - if(op == MPI_REPLACE) - { - *pop = PAMI_DATA_COPY; - return MPI_SUCCESS; - } - else return -1; - } - else if(isLOGICAL(dt)) - { - *pdt = PAMI_TYPE_LOGICAL4; - if(op == -1) return MPI_SUCCESS; - if(op == MPI_LOR) - { - *pop = PAMI_DATA_LOR; - return MPI_SUCCESS; - } - if(op == MPI_LAND) - { - *pop = PAMI_DATA_LAND; - return MPI_SUCCESS; - } - if(op == MPI_LXOR) - { - *pop = PAMI_DATA_LXOR; - return MPI_SUCCESS; - } - if(op == MPI_REPLACE) - { - *pop = PAMI_DATA_COPY; - return MPI_SUCCESS; - } - return -1; - } - else if(isBOOL(dt)) - { - *pdt = PAMI_TYPE_LOGICAL1; - if(op == -1) return MPI_SUCCESS; - if(op == MPI_LOR) - { - *pop = PAMI_DATA_LOR; - return MPI_SUCCESS; - } - if(op == MPI_LAND) - { - *pop = PAMI_DATA_LAND; - return MPI_SUCCESS; - } - if(op == MPI_LXOR) - { - *pop = PAMI_DATA_LXOR; - return MPI_SUCCESS; - } - if(op == MPI_REPLACE) - { - *pop = PAMI_DATA_COPY; - return MPI_SUCCESS; - } - return -1; - } - } - - if(*pdt == PAMI_TYPE_NULL) return -1; - - if(op == -1) return MPI_SUCCESS; /* just doing DT conversion */ - - *pop = PAMI_DATA_NOOP; - switch(op) - { - case MPI_SUM: *pop = PAMI_DATA_SUM; return MPI_SUCCESS; break; - case MPI_PROD: *pop = PAMI_DATA_PROD; return MPI_SUCCESS; break; - case MPI_MAX: *pop = PAMI_DATA_MAX; return MPI_SUCCESS; break; - case MPI_MIN: *pop = PAMI_DATA_MIN; return MPI_SUCCESS; break; - case MPI_BAND: *pop = PAMI_DATA_BAND; return MPI_SUCCESS; break; - case MPI_BOR: *pop = PAMI_DATA_BOR; return MPI_SUCCESS; break; - case MPI_BXOR: *pop = PAMI_DATA_BXOR; return MPI_SUCCESS; break; - case MPI_LAND: - if(isLONG_DOUBLE(dt)) return -1; - *pop = PAMI_DATA_LAND; return MPI_SUCCESS; break; - case MPI_LOR: - if(isLONG_DOUBLE(dt)) return -1; - *pop = PAMI_DATA_LOR; return MPI_SUCCESS; break; - case MPI_LXOR: - if(isLONG_DOUBLE(dt)) return -1; - *pop = PAMI_DATA_LXOR; return MPI_SUCCESS; break; - case MPI_REPLACE: *pop = PAMI_DATA_COPY; return MPI_SUCCESS; break; - } - if(*pop == PAMI_DATA_NOOP) return -1; - - return MPI_SUCCESS; -} - -int MPIDI_Dtpami_to_dtmpi( pami_type_t pdt, - MPI_Datatype *dt, - pami_data_function pop, - MPI_Op *op) -{ - *dt = MPI_DATATYPE_NULL; - if(pop != NULL) - *op = MPI_OP_NULL; - - if(pdt == PAMI_TYPE_SIGNED_INT) *dt = MPI_INT; - else if(pdt == PAMI_TYPE_UNSIGNED_INT) *dt = MPI_UNSIGNED; - else if(pdt == PAMI_TYPE_FLOAT) *dt = MPI_FLOAT; - else if(pdt == PAMI_TYPE_DOUBLE) *dt = MPI_DOUBLE; - else if(pdt == PAMI_TYPE_LONG_DOUBLE) *dt = MPI_LONG_DOUBLE; - else if(pdt == PAMI_TYPE_SIGNED_CHAR) *dt = MPI_CHAR; - else if(pdt == PAMI_TYPE_UNSIGNED_CHAR) *dt = MPI_UNSIGNED_CHAR; - else if(pdt == PAMI_TYPE_BYTE) *dt = MPI_BYTE; - else if(pdt == PAMI_TYPE_SIGNED_SHORT) *dt = MPI_SHORT; - else if(pdt == PAMI_TYPE_UNSIGNED_SHORT) *dt = MPI_UNSIGNED_SHORT; - else if(pdt == PAMI_TYPE_SIGNED_LONG_LONG) *dt = MPI_LONG_LONG; - else if(pdt == PAMI_TYPE_UNSIGNED_LONG_LONG) *dt = MPI_UNSIGNED_LONG_LONG; - else if(pdt == PAMI_TYPE_SINGLE_COMPLEX) *dt = MPI_C_FLOAT_COMPLEX; - else if(pdt == PAMI_TYPE_DOUBLE_COMPLEX) *dt = MPI_C_DOUBLE_COMPLEX; - else if(isPAMI_LOC_TYPE(pdt)) - { - if(pdt == PAMI_TYPE_LOC_2INT) *dt = MPI_2INT; - else if(pdt == PAMI_TYPE_LOC_FLOAT_INT) *dt = MPI_FLOAT_INT; - else if(pdt == PAMI_TYPE_LOC_DOUBLE_INT) *dt = MPI_DOUBLE_INT; - else if(pdt == PAMI_TYPE_LOC_SHORT_INT) *dt = MPI_SHORT_INT; - else if(pdt == PAMI_TYPE_LOC_LONG_INT) *dt = MPI_LONG_INT; - else if(pdt == PAMI_TYPE_LOC_LONGDOUBLE_INT)*dt = MPI_LONG_DOUBLE_INT; - else if(pdt == PAMI_TYPE_LOC_2FLOAT) *dt = MPI_2REAL; - else if(pdt == PAMI_TYPE_LOC_2DOUBLE) *dt = MPI_2DOUBLE_PRECISION; - - if(pop == NULL) return MPI_SUCCESS; - if(pop == PAMI_DATA_MINLOC) - { - *op = MPI_MINLOC; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_MAXLOC) - { - *op = MPI_MAXLOC; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_COPY) - { - *op = MPI_REPLACE; - return MPI_SUCCESS; - } - else return -1; - } - else if(pdt == PAMI_TYPE_LOGICAL4) - { - *dt = MPI_LOGICAL; - if(pop == NULL) return MPI_SUCCESS; - if(pop == PAMI_DATA_LOR) - { - *op = MPI_LOR; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_LAND) - { - *op = MPI_LAND; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_LXOR) - { - *op = MPI_LXOR; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_COPY) - { - *op = MPI_REPLACE; - return MPI_SUCCESS; - } - return -1; - } - else if(pdt == PAMI_TYPE_LOGICAL1) - { - *dt = MPI_C_BOOL; - if(pop == NULL) return MPI_SUCCESS; - if(pop == PAMI_DATA_LOR) - { - *op = MPI_LOR; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_LAND) - { - *op = MPI_LAND; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_LXOR) - { - *op = MPI_LXOR; - return MPI_SUCCESS; - } - if(pop == PAMI_DATA_COPY) - { - *op = MPI_REPLACE; - return MPI_SUCCESS; - } - return -1; - } - - - if(*dt == MPI_DATATYPE_NULL) return -1; - - if(pop == NULL) return MPI_SUCCESS; /* just doing DT conversion */ - - *op = MPI_OP_NULL; - if(pop == PAMI_DATA_SUM) *op = MPI_SUM; - else if(pop ==PAMI_DATA_PROD) *op = MPI_PROD; - else if(pop ==PAMI_DATA_MAX) *op = MPI_MAX; - else if(pop ==PAMI_DATA_MIN) *op = MPI_MIN; - else if(pop ==PAMI_DATA_BAND) *op = MPI_BAND; - else if(pop ==PAMI_DATA_BOR) *op = MPI_BOR; - else if(pop ==PAMI_DATA_BXOR) *op = MPI_BXOR; - else if(pop ==PAMI_DATA_LAND) *op = MPI_LAND; - else if(pop ==PAMI_DATA_LOR) *op = MPI_LOR; - else if(pop ==PAMI_DATA_LXOR) *op = MPI_LXOR; - else if(pop ==PAMI_DATA_COPY) *op = MPI_REPLACE; - - if(*op == MPI_OP_NULL) return -1; - - return MPI_SUCCESS; -} - diff --git a/src/mpid/pamid/src/coll/exscan/Makefile.mk b/src/mpid/pamid/src/coll/exscan/Makefile.mk deleted file mode 100644 index 511bc9d..0000000 --- a/src/mpid/pamid/src/coll/exscan/Makefile.mk +++ /dev/null @@ -1,28 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/exscan/mpido_iexscan.c - - -endif BUILD_PAMID diff --git a/src/mpid/pamid/src/coll/exscan/mpido_iexscan.c b/src/mpid/pamid/src/coll/exscan/mpido_iexscan.c deleted file mode 100644 index 2da2b08..0000000 --- a/src/mpid/pamid/src/coll/exscan/mpido_iexscan.c +++ /dev/null @@ -1,65 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/exscan/mpido_iexscan.c - * \brief ??? - */ - -/*#define TRACE_ON */ -#include <mpidimpl.h> - -int MPIDO_Iexscan(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, MPIR_Request **request) -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iexscan_impl() function to invoke the mpich nbc implementation - * of MPI_Iexscan(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iexscan() with a blocking MPIR_Exscan(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking exscan algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Exscan_impl(sendbuf, recvbuf, count, datatype, op, comm_ptr, &mpierrno); - - /* - * The blocking exscan has completed - create and complete a - * MPIR_Request object so the MPIR_Iexscan_impl() function does not - * perform an additional iexscan. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/gather/Makefile.mk b/src/mpid/pamid/src/coll/gather/Makefile.mk deleted file mode 100644 index 7b95dee..0000000 --- a/src/mpid/pamid/src/coll/gather/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/gather/mpido_gather.c \ - src/mpid/pamid/src/coll/gather/mpido_igather.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/gather/mpido_gather.c b/src/mpid/pamid/src/coll/gather/mpido_gather.c deleted file mode 100644 index c99265d..0000000 --- a/src/mpid/pamid/src/coll/gather/mpido_gather.c +++ /dev/null @@ -1,642 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gather/mpido_gather.c - * \brief ??? - */ - -#include <mpidimpl.h> - -static void cb_gather(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_gather enter, active: %u\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - -static void cb_allred(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_allred preallred enter, active: %u\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} -int MPIDO_Gather_reduce(void * sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - MPIDU_Datatype* data_ptr; - MPI_Aint true_lb ATTRIBUTE((unused)); - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - int rc, sbytes, rbytes; - int contig ATTRIBUTE((unused)); - char *tempbuf = NULL; - char *inplacetemp = NULL; - - MPIDI_Datatype_get_info(sendcount, sendtype, contig, - sbytes, data_ptr, true_lb); - - MPIDI_Datatype_get_info(recvcount, recvtype, contig, - rbytes, data_ptr, true_lb); - - if(rank == root) - { - tempbuf = recvbuf; - /* zero the buffer if we aren't in place */ - if(sendbuf != MPI_IN_PLACE) - { - memset(tempbuf, 0, sbytes * size * sizeof(char)); - memcpy(tempbuf+(rank * sbytes), sendbuf, sbytes); - } - /* copy our contribution temporarily, zero the buffer, put it back */ - /* this will likely suck */ - else - { - inplacetemp = MPL_malloc(rbytes * sizeof(char)); - memcpy(inplacetemp, recvbuf+(rank * rbytes), rbytes); - memset(tempbuf, 0, sbytes * size * sizeof(char)); - memcpy(tempbuf+(rank * rbytes), inplacetemp, rbytes); - MPL_free(inplacetemp); - } - } - /* everyone might need to speciifcally allocate a tempbuf, or - * we might need to make sure we don't end up at mpich in the - * mpido_() functions - seems to be a problem? - */ - - /* If we aren't root, malloc tempbuf and zero it, - * then copy our contribution to the right spot in the big buffer */ - else - { - tempbuf = MPL_malloc(sbytes * size * sizeof(char)); - if(!tempbuf) - return MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, - __FUNCTION__, - __LINE__, - MPI_ERR_OTHER, - "**nomem", 0); - - memset(tempbuf, 0, sbytes * size * sizeof(char)); - memcpy(tempbuf+(rank*sbytes), sendbuf, sbytes); - } - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Reduce(MPI_IN_PLACE, - tempbuf, - (sbytes * size)/4, - MPI_INT, - MPI_BOR, - root, - comm_ptr, - mpierrno); - - if(rank != root) - MPL_free(tempbuf); - - return rc; -} - -/* works for simple data types, assumes fast reduce is available */ - -int MPIDO_Gather(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype* data_ptr; - MPI_Aint true_lb ATTRIBUTE((unused)); - pami_xfer_t gather; - MPIDI_Post_coll_t gather_post; - int use_opt = 1, contig=0, send_bytes=-1, recv_bytes = 0; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_GATHER]; - - if (rank == root) - { - if (recvtype != MPI_DATATYPE_NULL && recvcount >= 0) - { - MPIDI_Datatype_get_info(recvcount, recvtype, contig, - recv_bytes, data_ptr, true_lb); - if (!contig || ((recv_bytes * size) % sizeof(int))) /* ? */ - use_opt = 0; - } - else - use_opt = 0; - } - - if ((sendbuf != MPI_IN_PLACE) && sendtype != MPI_DATATYPE_NULL && sendcount >= 0) - { - MPIDI_Datatype_get_info(sendcount, sendtype, contig, - send_bytes, data_ptr, true_lb); - if (!contig || ((send_bytes * size) % sizeof(int))) - use_opt = 0; - } - else - { - if(sendbuf == MPI_IN_PLACE) - send_bytes = recv_bytes; - if (sendtype == MPI_DATATYPE_NULL || sendcount == 0) - { - send_bytes = 0; - use_opt = 0; - } - } - - if(!mpid->optgather && - selected_type == MPID_COLL_USE_MPICH) - { - MPIDI_Update_last_algorithm(comm_ptr, "GATHER_MPICH"); - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH gather algorithm (01) opt %x, selected type %d\n",mpid->optgather,selected_type); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = (rank == root) ? MPIDI_cuda_is_device_buf(recvbuf) : 0; - if(is_send_dev_buf) - { - scbuf = MPL_malloc(sdt_extent * sendcount); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, sdt_extent * sendcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(rdt_extent * recvcount); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, rdt_extent * recvcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, rdt_extent * recvcount); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Gather(scbuf, sendcount, sendtype, rcbuf, recvcount, recvtype, root, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, rdt_extent * recvcount, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Gather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - if(mpid->preallreduces[MPID_GATHER_PREALLREDUCE]) - { - if(unlikely(verbose)) - fprintf(stderr,"MPID_GATHER_PREALLREDUCE opt %x, selected type %d, use_opt %d\n",mpid->optgather,selected_type, use_opt); - volatile unsigned allred_active = 1; - pami_xfer_t allred; - MPIDI_Post_coll_t allred_post; - allred.cb_done = cb_allred; - allred.cookie = (void *)&allred_active; - /* Guaranteed to work allreduce */ - allred.algorithm = mpid->coll_algorithm[PAMI_XFER_ALLREDUCE][0][0]; - allred.cmd.xfer_allreduce.sndbuf = (void *)PAMI_IN_PLACE; - allred.cmd.xfer_allreduce.stype = PAMI_TYPE_SIGNED_INT; - allred.cmd.xfer_allreduce.rcvbuf = (void *)&use_opt; - allred.cmd.xfer_allreduce.rtype = PAMI_TYPE_SIGNED_INT; - allred.cmd.xfer_allreduce.stypecount = 1; - allred.cmd.xfer_allreduce.rtypecount = 1; - allred.cmd.xfer_allreduce.op = PAMI_DATA_BAND; - - MPIDI_Context_post(MPIDI_Context[0], &allred_post.state, - MPIDI_Pami_post_wrapper, (void *)&allred); - MPID_PROGRESS_WAIT_WHILE(allred_active); - if(unlikely(verbose)) - fprintf(stderr,"MPID_GATHER_PREALLREDUCE opt %x, selected type %d, use_opt %d\n",mpid->optgather,selected_type, use_opt); - } - - if(mpid->optgather) - { - if(use_opt) - { - MPIDI_Update_last_algorithm(comm_ptr, "GLUE_REDUCDE"); - abort(); - /* GLUE_REDUCE ? */ - } - else - { - MPIDI_Update_last_algorithm(comm_ptr, "GATHER_MPICH"); - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH gather algorithm (02) opt %x, selected type %d, use_opt %d\n",mpid->optgather,selected_type, use_opt); - return MPIR_Gather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - } - - - pami_algorithm_t my_gather; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - volatile unsigned active = 1; - - gather.cb_done = cb_gather; - gather.cookie = (void *)&active; - gather.cmd.xfer_gather.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - if(sendbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"gather MPI_IN_PLACE buffering\n"); - gather.cmd.xfer_gather.stypecount = recv_bytes; - gather.cmd.xfer_gather.sndbuf = PAMI_IN_PLACE; - } - else - { - gather.cmd.xfer_gather.stypecount = send_bytes; - gather.cmd.xfer_gather.sndbuf = (void *)sendbuf; - } - gather.cmd.xfer_gather.stype = PAMI_TYPE_BYTE; - gather.cmd.xfer_gather.rcvbuf = (void *)recvbuf; - gather.cmd.xfer_gather.rtype = PAMI_TYPE_BYTE; - gather.cmd.xfer_gather.rtypecount = recv_bytes; - - /* If glue-level protocols are good, this will require some changes */ - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized gather (%s) was pre-selected\n", - mpid->opt_protocol_md[PAMI_XFER_GATHER][0].name); - my_gather = mpid->opt_protocol[PAMI_XFER_GATHER][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_GATHER][0]; - queryreq = mpid->must_query[PAMI_XFER_GATHER][0]; - } - else - { - TRACE_ERR("Optimized gather (%s) was specified by user\n", - mpid->user_metadata[PAMI_XFER_GATHER].name); - my_gather = mpid->user_selected[PAMI_XFER_GATHER]; - my_md = &mpid->user_metadata[PAMI_XFER_GATHER]; - queryreq = selected_type; - } - - gather.algorithm = my_gather; - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying gather protocol %s, type was %d\n", - my_md->name, queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; - if(my_md->check_correct.values.rangeminmax) - { - /* Non-local decision? */ - if(((rank == root) && - (my_md->range_lo <= recv_bytes) && - (my_md->range_hi >= recv_bytes) - ) && - ((rank != root) && - (my_md->range_lo <= send_bytes) && - (my_md->range_hi >= send_bytes) - ) - ) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - recv_bytes, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&gather); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"query failed for %s\n", my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "GATHER_MPICH"); - return MPIR_Gather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - MPIDI_Update_last_algorithm(comm_ptr, - mpid->user_metadata[PAMI_XFER_GATHER].name); - - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for gather on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - - MPIDI_Context_post(MPIDI_Context[0], &gather_post.state, - MPIDI_Pami_post_wrapper, (void *)&gather); - - TRACE_ERR("Waiting on active: %d\n", active); - MPID_PROGRESS_WAIT_WHILE(active); - - TRACE_ERR("Leaving MPIDO_Gather\n"); - return 0; -} - - -int MPIDO_Gather_simple(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype* data_ptr; - MPI_Aint true_lb = 0; - pami_xfer_t gather; - MPIDI_Post_coll_t gather_post; - int success = 1, snd_contig = 1, rcv_contig = 1; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - int send_size = 0; - int recv_size = 0; - MPIDU_Segment segment; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - if(sendbuf != MPI_IN_PLACE) - { - MPIDI_Datatype_get_info(sendcount, sendtype, snd_contig, - send_size, data_ptr, true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_GATHER, send_size, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Gather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - sbuf = (char *)sendbuf + true_lb; - if (!snd_contig) - { - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - DLOOP_Offset last = send_size; - MPIDU_Segment_init(sendbuf, sendcount, sendtype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, snd_noncontig_buff); - } - } - else - { - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - MPIDI_Datatype_get_info(recvcount, recvtype, rcv_contig, - recv_size, data_ptr, true_lb); - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_GATHER, recv_size, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Gather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - - } - } - } - - rbuf = (char *)recvbuf + true_lb; - if (rank == root) - { - if (recvtype != MPI_DATATYPE_NULL && recvcount >= 0) - { - MPIDI_Datatype_get_info(recvcount, recvtype, rcv_contig, - recv_size, data_ptr, true_lb); - rbuf = (char *)recvbuf + true_lb; - if (!rcv_contig) - { - rcv_noncontig_buff = MPL_malloc(recv_size * size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendbuf == MPI_IN_PLACE) - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - MPIR_Localcopy(recvbuf + (rank*recvcount*extent), recvcount, recvtype, - rcv_noncontig_buff + (rank*recv_size), recv_size,MPI_CHAR); - } - } - } - else - success = 0; - } - - if(!success) - { - MPIDI_Update_last_algorithm(comm_ptr, "GATHER_MPICH"); - return MPIR_Gather(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - - - const pami_metadata_t *my_gather_md; - volatile unsigned active = 1; - - gather.cb_done = cb_gather; - gather.cookie = (void *)&active; - gather.cmd.xfer_gather.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - gather.cmd.xfer_gather.stypecount = send_size;/* stypecount is ignored when sndbuf == PAMI_IN_PLACE */ - gather.cmd.xfer_gather.sndbuf = (void *)sbuf; - if(sendbuf == MPI_IN_PLACE) - { - gather.cmd.xfer_gather.sndbuf = PAMI_IN_PLACE; - } - gather.cmd.xfer_gather.stype = PAMI_TYPE_BYTE;/* stype is ignored when sndbuf == PAMI_IN_PLACE */ - gather.cmd.xfer_gather.rcvbuf = (void *)rbuf; - gather.cmd.xfer_gather.rtype = PAMI_TYPE_BYTE; - gather.cmd.xfer_gather.rtypecount = recv_size; - gather.algorithm = mpid->coll_algorithm[PAMI_XFER_GATHER][0][0]; - my_gather_md = &mpid->coll_metadata[PAMI_XFER_GATHER][0][0]; - - MPIDI_Update_last_algorithm(comm_ptr, - my_gather_md->name); - - TRACE_ERR("%s gather\n", MPIDI_Process.context_post.active>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &gather_post.state, - MPIDI_Pami_post_wrapper, (void *)&gather); - - TRACE_ERR("Waiting on active: %d\n", active); - MPID_PROGRESS_WAIT_WHILE(active); - - if(!rcv_contig) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size*size, MPI_CHAR, - recvbuf, recvcount*size, recvtype); - MPL_free(rcv_noncontig_buff); - } - if(!snd_contig) MPL_free(snd_noncontig_buff); - - TRACE_ERR("Leaving MPIDO_Gather_optimized\n"); - return MPI_SUCCESS; -} - -int -MPIDO_CSWrapper_gather(pami_xfer_t *gather, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *sbuf; - MPIDI_coll_check_in_place(gather->cmd.xfer_gather.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( gather->cmd.xfer_gather.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( gather->cmd.xfer_gather.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Gather(sbuf, - gather->cmd.xfer_gather.stypecount, sendtype, - gather->cmd.xfer_gather.rcvbuf, - gather->cmd.xfer_gather.rtypecount, recvtype, - gather->cmd.xfer_gather.root, comm_ptr, &mpierrno); - - if(gather->cb_done && rc == 0) - gather->cb_done(NULL, gather->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/gather/mpido_igather.c b/src/mpid/pamid/src/coll/gather/mpido_igather.c deleted file mode 100644 index f55eea4..0000000 --- a/src/mpid/pamid/src/coll/gather/mpido_igather.c +++ /dev/null @@ -1,72 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gather/mpido_igather.c - * \brief ??? - */ - -#include <mpidimpl.h> - -int MPIDO_Igather(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Igather_impl() function to invoke the mpich nbc implementation - * of MPI_Igather(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Igather() with a blocking MPIR_Gather(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking gather algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Gather_impl(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, &mpierrno); - - /* - * The blocking gather has completed - create and complete a - * MPIR_Request object so the MPIR_Igather_impl() function does not - * perform an additional igather. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/gatherv/Makefile.mk b/src/mpid/pamid/src/coll/gatherv/Makefile.mk deleted file mode 100644 index ee401df..0000000 --- a/src/mpid/pamid/src/coll/gatherv/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/gatherv/mpido_gatherv.c \ - src/mpid/pamid/src/coll/gatherv/mpido_igatherv.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/gatherv/mpido_gatherv.c b/src/mpid/pamid/src/coll/gatherv/mpido_gatherv.c deleted file mode 100644 index 5832468..0000000 --- a/src/mpid/pamid/src/coll/gatherv/mpido_gatherv.c +++ /dev/null @@ -1,573 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gather/mpido_gatherv.c - * \brief ??? - */ - -/* #define TRACE_ON */ -#include <mpidimpl.h> - -static void cb_gatherv(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_gatherv enter, active: %u\n", (*active)); - (*active)--; -} - -int MPIDO_Gatherv(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *displs, - MPI_Datatype recvtype, - int root, - MPIR_Comm * comm_ptr, - int *mpierrno) - -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Gatherv\n"); - int i; - int contig ATTRIBUTE((unused)), rsize ATTRIBUTE((unused)), ssize ATTRIBUTE((unused)); - int pamidt = 1; - MPIDU_Datatype*dt_ptr = NULL; - MPI_Aint send_true_lb, recv_true_lb; - char *sbuf, *rbuf; - pami_type_t stype, rtype; - int tmp; - volatile unsigned gatherv_active = 1; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_GATHERV_INT]; - - /* Check for native PAMI types and MPI_IN_PLACE on sendbuf */ - /* MPI_IN_PLACE is a nonlocal decision. We will need a preallreduce if we ever have - * multiple "good" gathervs that work on different counts for example */ - if((sendbuf != MPI_IN_PLACE) && (MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp) != MPI_SUCCESS)) - pamidt = 0; - if(MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp) != MPI_SUCCESS) - pamidt = 0; - - if(pamidt == 0 || selected_type == MPID_COLL_USE_MPICH) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH gatherv algorithm\n"); - TRACE_ERR("GATHERV using MPICH\n"); - MPIDI_Update_last_algorithm(comm_ptr, "GATHERV_MPICH"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = (rank == root) ? MPIDI_cuda_is_device_buf(recvbuf) : 0; - if(is_send_dev_buf) - { - scbuf = MPL_malloc(sdt_extent * sendcount); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, sdt_extent * sendcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - size_t rtotal_buf; - if(is_recv_dev_buf) - { - //Since displs can be non-continous, we need to calculate max buffer size - int highest_displs = displs[size - 1]; - int highest_recvcount = recvcounts[size - 1]; - for(i = 0; i < size; i++) - { - if(displs[i]+recvcounts[i] > highest_displs+highest_recvcount) - { - highest_displs = displs[i]; - highest_recvcount = recvcounts[i]; - } - } - rtotal_buf = (highest_displs+highest_recvcount)*rdt_extent; - rcbuf = MPL_malloc(rtotal_buf); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, rtotal_buf, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, rtotal_buf); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Gatherv(scbuf, sendcount, sendtype, rcbuf, recvcounts, displs, recvtype, root, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, rtotal_buf, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Gatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - root, comm_ptr, mpierrno); - } - - MPIDI_Datatype_get_info(1, recvtype, contig, rsize, dt_ptr, recv_true_lb); - rbuf = (char *)recvbuf + recv_true_lb; - sbuf = (void *) sendbuf; - - pami_xfer_t gatherv; - - gatherv.cb_done = cb_gatherv; - gatherv.cookie = (void *)&gatherv_active; - gatherv.cmd.xfer_gatherv_int.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - gatherv.cmd.xfer_gatherv_int.rcvbuf = rbuf; - gatherv.cmd.xfer_gatherv_int.rtype = rtype; - gatherv.cmd.xfer_gatherv_int.rtypecounts = (int *) recvcounts; - gatherv.cmd.xfer_gatherv_int.rdispls = (int *) displs; - - gatherv.cmd.xfer_gatherv_int.sndbuf = NULL; - gatherv.cmd.xfer_gatherv_int.stype = stype; - gatherv.cmd.xfer_gatherv_int.stypecount = sendcount; - - if(rank == root) - { - if(sendbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"gatherv MPI_IN_PLACE buffering\n"); - sbuf = PAMI_IN_PLACE; - gatherv.cmd.xfer_gatherv_int.stype = rtype; - gatherv.cmd.xfer_gatherv_int.stypecount = recvcounts[rank]; - } - else - { - MPIDI_Datatype_get_info(1, sendtype, contig, ssize, dt_ptr, send_true_lb); - sbuf = (char *)sbuf + send_true_lb; - } - } - gatherv.cmd.xfer_gatherv_int.sndbuf = sbuf; - - pami_algorithm_t my_gatherv; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized gatherv %s was selected\n", - mpid->opt_protocol_md[PAMI_XFER_GATHERV_INT][0].name); - my_gatherv = mpid->opt_protocol[PAMI_XFER_GATHERV_INT][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_GATHERV_INT][0]; - queryreq = mpid->must_query[PAMI_XFER_GATHERV_INT][0]; - } - else - { - TRACE_ERR("Optimized gatherv %s was set by user\n", - mpid->user_metadata[PAMI_XFER_GATHERV_INT].name); - my_gatherv = mpid->user_selected[PAMI_XFER_GATHERV_INT]; - my_md = &mpid->user_metadata[PAMI_XFER_GATHERV_INT]; - queryreq = selected_type; - } - - gatherv.algorithm = my_gatherv; - - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying gatherv protocol %s, type was %d\n", - my_md->name, queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; -/* Can't check ranges like this. Non-local. Comment out for now. - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb; - MPIDU_Datatype*data_ptr; - int data_size, data_contig; - MPIDI_Datatype_get_info(sendcount, sendtype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - */ - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&gatherv); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH gatherv.\n", my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "GATHERV_MPICH"); - return MPIR_Gatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - root, comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for gatherv on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - - MPIDI_Post_coll_t gatherv_post; - MPIDI_Context_post(MPIDI_Context[0], &gatherv_post.state, - MPIDI_Pami_post_wrapper, (void *)&gatherv); - - TRACE_ERR("Waiting on active %d\n", gatherv_active); - MPID_PROGRESS_WAIT_WHILE(gatherv_active); - - TRACE_ERR("Leaving MPIDO_Gatherv\n"); - return 0; -} - - -int MPIDO_Gatherv_simple(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *displs, - MPI_Datatype recvtype, - int root, - MPIR_Comm * comm_ptr, - int *mpierrno) - -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - TRACE_ERR("Entering MPIDO_Gatherv_optimized\n"); - int snd_contig = 1, rcv_contig = 1; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - int *rcounts = NULL; - int *rdispls = NULL; - int send_size = 0; - int recv_size = 0; - int rcvlen = 0; - int totalrecvcount = 0; - pami_type_t rtype = PAMI_TYPE_NULL; - MPIDU_Segment segment; - MPIDU_Datatype*data_ptr = NULL; - int send_true_lb, recv_true_lb = 0; - int i, tmp; - volatile unsigned gatherv_active = 1; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - int recvok=PAMI_SUCCESS, recvcontinuous=0; - - if(sendbuf != MPI_IN_PLACE) - { - MPIDI_Datatype_get_info(sendcount, sendtype, snd_contig, - send_size, data_ptr, send_true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_GATHERV_INT, 64, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Gatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - sbuf = (char *)sendbuf + send_true_lb; - if(!snd_contig) - { - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - DLOOP_Offset last = send_size; - MPIDU_Segment_init(sendbuf, sendcount, sendtype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, snd_noncontig_buff); - } - } - else - { - MPIDI_Datatype_get_info(1, recvtype, rcv_contig, - rcvlen, data_ptr, recv_true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_GATHERV_INT, 64, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Gatherv(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - } - - pami_xfer_t gatherv; - rbuf = (char *)recvbuf + recv_true_lb; - rcounts = (int*)recvcounts; - rdispls = (int*)displs; - if(rank == root) - { - if((recvok = MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp)) != MPI_SUCCESS) - { - MPIDI_Datatype_get_info(1, recvtype, rcv_contig, - rcvlen, data_ptr, recv_true_lb); - totalrecvcount = recvcounts[0]; - recvcontinuous = displs[0] == 0? 1 : 0 ; - rcounts = (int*)MPL_malloc(size); - rdispls = (int*)MPL_malloc(size); - rdispls[0] = 0; - rcounts[0] = rcvlen * recvcounts[0]; - for(i = 1; i < size; i++) - { - rdispls[i]= rcvlen * totalrecvcount; - totalrecvcount += recvcounts[i]; - if(displs[i] != (displs[i-1] + recvcounts[i-1])) - recvcontinuous = 0; - rcounts[i] = rcvlen * recvcounts[i]; - } - recv_size = rcvlen * totalrecvcount; - - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - rtype = PAMI_TYPE_BYTE; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - if(sendbuf == MPI_IN_PLACE) - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - MPIR_Localcopy(recvbuf + displs[rank]*extent, recvcounts[rank], recvtype, - rcv_noncontig_buff + rdispls[rank], rcounts[rank],MPI_CHAR); - } - } - if(sendbuf == MPI_IN_PLACE) - { - gatherv.cmd.xfer_gatherv_int.sndbuf = PAMI_IN_PLACE; - } - else - { - gatherv.cmd.xfer_gatherv_int.sndbuf = sbuf; - } - gatherv.cmd.xfer_gatherv_int.stype = PAMI_TYPE_BYTE;/* stype is ignored when sndbuf == PAMI_IN_PLACE */ - gatherv.cmd.xfer_gatherv_int.stypecount = send_size; - - } - else - { - gatherv.cmd.xfer_gatherv_int.sndbuf = sbuf; - gatherv.cmd.xfer_gatherv_int.stype = PAMI_TYPE_BYTE; - gatherv.cmd.xfer_gatherv_int.stypecount = send_size; - } - - - gatherv.cb_done = cb_gatherv; - gatherv.cookie = (void *)&gatherv_active; - gatherv.cmd.xfer_gatherv_int.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - gatherv.cmd.xfer_gatherv_int.rcvbuf = rbuf; - gatherv.cmd.xfer_gatherv_int.rtype = rtype; - gatherv.cmd.xfer_gatherv_int.rtypecounts = (int *) rcounts; - gatherv.cmd.xfer_gatherv_int.rdispls = (int *) rdispls; - - - const pami_metadata_t *my_gatherv_md; - - gatherv.algorithm = mpid->coll_algorithm[PAMI_XFER_GATHERV_INT][0][0]; - my_gatherv_md = &mpid->coll_metadata[PAMI_XFER_GATHERV_INT][0][0]; - - MPIDI_Update_last_algorithm(comm_ptr, my_gatherv_md->name); - - MPIDI_Post_coll_t gatherv_post; - TRACE_ERR("%s gatherv\n", MPIDI_Process.context_post.active>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &gatherv_post.state, - MPIDI_Pami_post_wrapper, (void *)&gatherv); - TRACE_ERR("Gatherv %s\n", MPIDI_Process.context_post.active>0?"posted":"invoked"); - - TRACE_ERR("Waiting on active %d\n", gatherv_active); - MPID_PROGRESS_WAIT_WHILE(gatherv_active); - - if(!rcv_contig || recvok != PAMI_SUCCESS) - { - if(recvcontinuous) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size, MPI_CHAR, - recvbuf, totalrecvcount, recvtype); - } - else - { - size_t extent; - MPIDU_Datatype_get_extent_macro(recvtype,extent); - for(i=0; i<size; ++i) - { - char* scbuf = (char*)rcv_noncontig_buff+ rdispls[i]; - char* rcbuf = (char*)recvbuf + displs[i]*extent; - MPIR_Localcopy(scbuf, rcounts[i], MPI_CHAR, - rcbuf, recvcounts[i], recvtype); - TRACE_ERR("Pack recv src extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)precvdispls[i],(size_t)i,(size_t)precvcounts[i],(size_t)precvdispls[i], *(int*)scbuf); - TRACE_ERR("Pack recv dest extent %zu, displ[%zu]=%zu, count[%zu]=%zu buf[%zu]=%u\n", - (size_t)extent, (size_t)i,(size_t)displs[i],(size_t)i,(size_t)recvcounts[i],(size_t)displs[i], *(int*)rcbuf); - } - - } - MPL_free(rcv_noncontig_buff); - if(rank == root) - { - MPL_free(rcounts); - MPL_free(rdispls); - } - } - if(!snd_contig) MPL_free(snd_noncontig_buff); - - - TRACE_ERR("Leaving MPIDO_Gatherv_optimized\n"); - return MPI_SUCCESS; -} - -int -MPIDO_CSWrapper_gatherv(pami_xfer_t *gatherv, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *sbuf; - MPIDI_coll_check_in_place(gatherv->cmd.xfer_gatherv_int.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( gatherv->cmd.xfer_gatherv_int.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - if(gatherv->cmd.xfer_gatherv_int.rtype == PAMI_TYPE_NULL) - recvtype = MPI_DATATYPE_NULL; - else - rc = MPIDI_Dtpami_to_dtmpi( gatherv->cmd.xfer_gatherv_int.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Gatherv(sbuf, - gatherv->cmd.xfer_gatherv_int.stypecount, sendtype, - gatherv->cmd.xfer_gatherv_int.rcvbuf, - gatherv->cmd.xfer_gatherv_int.rtypecounts, - gatherv->cmd.xfer_gatherv_int.rdispls, recvtype, - gatherv->cmd.xfer_gatherv_int.root, comm_ptr, &mpierrno); - if(gatherv->cb_done && rc == 0) - gatherv->cb_done(NULL, gatherv->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/gatherv/mpido_igatherv.c b/src/mpid/pamid/src/coll/gatherv/mpido_igatherv.c deleted file mode 100644 index 3a315d6..0000000 --- a/src/mpid/pamid/src/coll/gatherv/mpido_igatherv.c +++ /dev/null @@ -1,78 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gatherv/mpido_igatherv.c - * \brief ??? - */ - -/*#define TRACE_ON*/ -#include <mpidimpl.h> - -int MPIDO_Igatherv(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - const int *recvcounts, - const int *displs, - MPI_Datatype recvtype, - int root, - MPIR_Comm * comm_ptr, - MPIR_Request **request) - -{ - TRACE_ERR("Entering MPIDO_Igatherv\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Igatherv_impl() function to invoke the mpich nbc implementation - * of MPI_Igatherv(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Igatherv() with a blocking MPIR_Gatherv(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking gatherv algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Gatherv_impl(sendbuf, sendcount, sendtype, - recvbuf, recvcounts, displs, recvtype, - root, comm_ptr, &mpierrno); - - /* - * The blocking gather has completed - create and complete a - * MPIR_Request object so the MPIR_Igatherv_impl() function does not - * perform an additional igatherv. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - - return 0; -} diff --git a/src/mpid/pamid/src/coll/ired_scat/Makefile.mk b/src/mpid/pamid/src/coll/ired_scat/Makefile.mk deleted file mode 100644 index b975ffd..0000000 --- a/src/mpid/pamid/src/coll/ired_scat/Makefile.mk +++ /dev/null @@ -1,28 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/ired_scat/mpido_ired_scat.c - - -endif BUILD_PAMID diff --git a/src/mpid/pamid/src/coll/ired_scat/mpido_ired_scat.c b/src/mpid/pamid/src/coll/ired_scat/mpido_ired_scat.c deleted file mode 100644 index 80d54d8..0000000 --- a/src/mpid/pamid/src/coll/ired_scat/mpido_ired_scat.c +++ /dev/null @@ -1,72 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/ired_scat/mpido_ired_scat.c - * \brief ??? - */ - -/*#define TRACE_ON*/ -#include <mpidimpl.h> - -int MPIDO_Ireduce_scatter(const void *sendbuf, - void *recvbuf, - const int *recvcounts, - MPI_Datatype datatype, - MPI_Op op, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - TRACE_ERR("Entering MPIDO_Ireduce_scatter\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Ireduce_scatter_impl() function to invoke the mpich nbc - * implementation of MPI_Ireduce_scatter(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ireduce_scatter() with a blocking MPIR_Reduce_scatter_block(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking reduce_scatter algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Reduce_scatter_impl(sendbuf, recvbuf, recvcounts, datatype, - op, comm_ptr, &mpierrno); - - /* - * The blocking gather has completed - create and complete a - * MPIR_Request object so the MPIR_Ireduce_scatter_impl() function - * does not perform an additional ireduce_scatter. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/ired_scat_block/Makefile.mk b/src/mpid/pamid/src/coll/ired_scat_block/Makefile.mk deleted file mode 100644 index 562c719..0000000 --- a/src/mpid/pamid/src/coll/ired_scat_block/Makefile.mk +++ /dev/null @@ -1,28 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/ired_scat_block/mpido_ired_scat_block.c - - -endif BUILD_PAMID diff --git a/src/mpid/pamid/src/coll/ired_scat_block/mpido_ired_scat_block.c b/src/mpid/pamid/src/coll/ired_scat_block/mpido_ired_scat_block.c deleted file mode 100644 index 1a9f738..0000000 --- a/src/mpid/pamid/src/coll/ired_scat_block/mpido_ired_scat_block.c +++ /dev/null @@ -1,72 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/ired_scat_block/mpido_ired_scat_block.c - * \brief ??? - */ - -/*#define TRACE_ON*/ -#include <mpidimpl.h> - -int MPIDO_Ireduce_scatter_block(const void *sendbuf, - void *recvbuf, - int recvcount, - MPI_Datatype datatype, - MPI_Op op, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - TRACE_ERR("Entering MPIDO_Ireduce_scatter_block\n"); - - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Ireduce_scatter_block_impl() function to invoke the mpich nbc - * implementation of MPI_Ireduce_scatter_block(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ireduce_scatter_block() with a blocking MPIR_Reduce_scatter_block(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking reduce_scatter_block algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Reduce_scatter_block_impl(sendbuf, recvbuf, recvcount, datatype, - op, comm_ptr, &mpierrno); - - /* - * The blocking gather has completed - create and complete a - * MPIR_Request object so the MPIR_Ireduce_scatter_block_impl() function - * does not perform an additional ireduce_scatter_block. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/red_scat/Makefile.mk b/src/mpid/pamid/src/coll/red_scat/Makefile.mk deleted file mode 100644 index a95beaf..0000000 --- a/src/mpid/pamid/src/coll/red_scat/Makefile.mk +++ /dev/null @@ -1,29 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/red_scat/mpido_red_scat.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/red_scat/mpido_red_scat.c b/src/mpid/pamid/src/coll/red_scat/mpido_red_scat.c deleted file mode 100644 index d207f45..0000000 --- a/src/mpid/pamid/src/coll/red_scat/mpido_red_scat.c +++ /dev/null @@ -1,185 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/red_scat/mpido_red_scat.c - * \brief ??? - */ - -/* #define TRACE_ON */ -#include <mpidimpl.h> - - -int MPIDO_Reduce_scatter(const void *sendbuf, - void *recvbuf, - int *recvcounts, - MPI_Datatype datatype, - MPI_Op op, - MPIR_Comm *comm_ptr, - int *mpierrno) - -{ - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH reduce_scatter algorithm\n"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint dt_extent; - MPIDU_Datatype_get_extent_macro(datatype, dt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - int i; - size_t total_buf = 0; - for(i = 0; i < size; i++) - { - total_buf += recvcounts[i]; - } - - if(is_send_dev_buf) - { - scbuf = MPL_malloc(dt_extent * total_buf); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, dt_extent * total_buf, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(total_buf * dt_extent); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, dt_extent * total_buf, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, total_buf * dt_extent); - } - else - rcbuf = recvbuf; - - int cuda_res = MPIR_Reduce_scatter(scbuf, rcbuf, recvcounts, datatype, op, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, dt_extent * total_buf, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Reduce_scatter(sendbuf, recvbuf, recvcounts, datatype, op, comm_ptr, mpierrno); - -} - - - -int MPIDO_Reduce_scatter_block(const void *sendbuf, - void *recvbuf, - int recvcount, - MPI_Datatype datatype, - MPI_Op op, - MPIR_Comm *comm_ptr, - int *mpierrno) - -{ - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH reduce_scatter algorithm\n"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint dt_extent; - MPIDU_Datatype_get_extent_macro(datatype, dt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - int i; - if(is_send_dev_buf) - { - scbuf = MPL_malloc(dt_extent * recvcount * size); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, dt_extent * recvcount * size, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s recvbuf: %p scbuf: %p is_send_dev_buf: %d is_recv_dev_buf: %p sendbuf: %p\n", CudaGetErrorString(cudaerr), recvbuf, scbuf, is_send_dev_buf,is_recv_dev_buf, sendbuf ); - } - else - scbuf = sendbuf; - - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(dt_extent * recvcount * size); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, dt_extent * recvcount * size, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, recvcount * size * dt_extent); - } - else - rcbuf = recvbuf; - - int cuda_res; - if(comm_ptr->comm_kind == MPIR_COMM_KIND__INTRACOMM) - cuda_res = MPIR_Reduce_scatter_block_intra(scbuf, rcbuf, recvcount, datatype, op, comm_ptr, mpierrno); - else - cuda_res = MPIR_Reduce_scatter_block_inter(scbuf, rcbuf, recvcount, datatype, op, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, dt_extent * recvcount * size, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s recvbuf: %p rcbuf: %p is_send_dev_buf: %d is_recv_dev_buf: %p sendbuf: %p\n", CudaGetErrorString(cudaerr), recvbuf, rcbuf, is_send_dev_buf,is_recv_dev_buf, sendbuf ); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - if(comm_ptr->comm_kind == MPIR_COMM_KIND__INTRACOMM) - return MPIR_Reduce_scatter_block_intra(sendbuf, recvbuf, recvcount, datatype, op, comm_ptr, mpierrno); - else - return MPIR_Reduce_scatter_block_inter(sendbuf, recvbuf, recvcount, datatype, op, comm_ptr, mpierrno); - -} - - diff --git a/src/mpid/pamid/src/coll/reduce/Makefile.mk b/src/mpid/pamid/src/coll/reduce/Makefile.mk deleted file mode 100644 index c962c91..0000000 --- a/src/mpid/pamid/src/coll/reduce/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/reduce/mpido_reduce.c \ - src/mpid/pamid/src/coll/reduce/mpido_ireduce.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/reduce/mpido_ireduce.c b/src/mpid/pamid/src/coll/reduce/mpido_ireduce.c deleted file mode 100644 index e7f7c57..0000000 --- a/src/mpid/pamid/src/coll/reduce/mpido_ireduce.c +++ /dev/null @@ -1,72 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gather/mpido_ireduce.c - * \brief ??? - */ - -/*#define TRACE_ON*/ -#include <mpidimpl.h> - -int MPIDO_Ireduce(const void *sendbuf, - void *recvbuf, - int count, - MPI_Datatype datatype, - MPI_Op op, - int root, - MPIR_Comm *comm_ptr, - MPIR_Request **request) - -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Ireduce_impl() function to invoke the mpich nbc - * implementation of MPI_Ireduce(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Ireduce() with a blocking MPIR_Reduce(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking reduce_algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Reduce_impl(sendbuf, recvbuf, count, datatype, - op, root, comm_ptr, &mpierrno); - - /* - * The blocking reduce has completed - create and complete a - * MPIR_Request object so the MPIR_Ireduce_impl() function - * does not perform an additional ireduce. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/reduce/mpido_reduce.c b/src/mpid/pamid/src/coll/reduce/mpido_reduce.c deleted file mode 100644 index b548a63..0000000 --- a/src/mpid/pamid/src/coll/reduce/mpido_reduce.c +++ /dev/null @@ -1,422 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gather/mpido_reduce.c - * \brief ??? - */ - -/* #define TRACE_ON */ -#include <mpidimpl.h> - -static void reduce_cb_done(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_reduce enter, active: %u\n", (*active)); - (*active)--; -} - -int MPIDO_Reduce(const void *sendbuf, - void *recvbuf, - int count, - MPI_Datatype datatype, - MPI_Op op, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) - -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype*dt_null = NULL; - MPI_Aint true_lb = 0; - int dt_contig ATTRIBUTE((unused)), tsize; - int mu; - char *sbuf, *rbuf; - pami_data_function pop; - pami_type_t pdt; - int rc; - int alg_selected = 0; - const int rank = comm_ptr->rank; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_REDUCE]; - - rc = MPIDI_Datatype_to_pami(datatype, &pdt, op, &pop, &mu); - if(unlikely(verbose)) - fprintf(stderr,"reduce - rc %u, root %u, count %d, dt: %p, op: %p, mu: %u, selectedvar %u != %u (MPICH) sendbuf %p, recvbuf %p\n", - rc, root, count, pdt, pop, mu, - (unsigned)selected_type, MPID_COLL_USE_MPICH,sendbuf, recvbuf); - - pami_xfer_t reduce; - pami_algorithm_t my_reduce=0; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; - volatile unsigned reduce_active = 1; - - MPIDI_Datatype_get_info(count, datatype, dt_contig, tsize, dt_null, true_lb); - rbuf = (char *)recvbuf + true_lb; - sbuf = (char *)sendbuf + true_lb; - if(sendbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"reduce MPI_IN_PLACE send buffering (%d,%d)\n",count,tsize); - sbuf = PAMI_IN_PLACE; - } - - reduce.cb_done = reduce_cb_done; - reduce.cookie = (void *)&reduce_active; - if(mpid->optreduce) /* GLUE_ALLREDUCE */ - { - char* tbuf = NULL; - if(unlikely(verbose)) - fprintf(stderr,"Using protocol GLUE_ALLREDUCE for reduce (%d,%d)\n",count,tsize); - MPIDI_Update_last_algorithm(comm_ptr, "REDUCE_OPT_ALLREDUCE"); - void *destbuf = recvbuf; - if(rank != root) /* temp buffer for non-root destbuf */ - { - tbuf = destbuf = MPL_malloc(tsize); - } - /* Switch to comm->coll_fns->fn() */ - MPIDO_Allreduce(sendbuf, - destbuf, - count, - datatype, - op, - comm_ptr, - mpierrno); - if(tbuf) - MPL_free(tbuf); - return 0; - } - if(selected_type == MPID_COLL_USE_MPICH || rc != MPI_SUCCESS) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH reduce algorithm\n"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint dt_extent; - MPIDU_Datatype_get_extent_macro(datatype, dt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(dt_extent * count); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, dt_extent * count, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(dt_extent * count); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, dt_extent * count, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, dt_extent * count); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Reduce(scbuf, rcbuf, count, datatype, op, root, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, dt_extent * count, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Reduce(sendbuf, recvbuf, count, datatype, op, root, comm_ptr, mpierrno); - } - - if(selected_type == MPID_COLL_OPTIMIZED) - { - if((mpid->cutoff_size[PAMI_XFER_REDUCE][0] == 0) || - (mpid->cutoff_size[PAMI_XFER_REDUCE][0] >= tsize && mpid->cutoff_size[PAMI_XFER_REDUCE][0] > 0)) - { - TRACE_ERR("Optimized Reduce (%s) was pre-selected\n", - mpid->opt_protocol_md[PAMI_XFER_REDUCE][0].name); - my_reduce = mpid->opt_protocol[PAMI_XFER_REDUCE][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_REDUCE][0]; - queryreq = mpid->must_query[PAMI_XFER_REDUCE][0]; - } - - } - else - { - TRACE_ERR("Optimized reduce (%s) was specified by user\n", - mpid->user_metadata[PAMI_XFER_REDUCE].name); - my_reduce = mpid->user_selected[PAMI_XFER_REDUCE]; - my_md = &mpid->user_metadata[PAMI_XFER_REDUCE]; - queryreq = selected_type; - } - reduce.algorithm = my_reduce; - reduce.cmd.xfer_reduce.sndbuf = sbuf; - reduce.cmd.xfer_reduce.rcvbuf = rbuf; - reduce.cmd.xfer_reduce.stype = pdt; - reduce.cmd.xfer_reduce.rtype = pdt; - reduce.cmd.xfer_reduce.stypecount = count; - reduce.cmd.xfer_reduce.rtypecount = count; - reduce.cmd.xfer_reduce.op = pop; - reduce.cmd.xfer_reduce.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("Querying reduce protocol %s, type was %d\n", - my_md->name, - queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb ATTRIBUTE((unused)); - MPIDU_Datatype*data_ptr; - int data_size, data_contig ATTRIBUTE((unused)); - MPIDI_Datatype_get_info(count, datatype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&reduce); - TRACE_ERR("Bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH reduce.\n", - my_md->name); - } - else - { - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - alg_selected = 1; - } - } - - if(alg_selected) - { - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for reduce on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - MPIDI_Post_coll_t reduce_post; - MPIDI_Context_post(MPIDI_Context[0], &reduce_post.state, - MPIDI_Pami_post_wrapper, (void *)&reduce); - } - else - { - MPIDI_Update_last_algorithm(comm_ptr, "REDUCE_MPICH"); - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH reduce algorithm\n"); - return MPIR_Reduce(sendbuf, recvbuf, count, datatype, op, root, comm_ptr, mpierrno); - } - - MPIDI_Update_last_algorithm(comm_ptr, - my_md->name); - MPID_PROGRESS_WAIT_WHILE(reduce_active); - TRACE_ERR("Reduce done\n"); - return 0; -} - - -int MPIDO_Reduce_simple(const void *sendbuf, - void *recvbuf, - int count, - MPI_Datatype datatype, - MPI_Op op, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) - -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype*dt_null = NULL; - MPI_Aint true_lb = 0; - int dt_contig, tsize; - int mu; - char *sbuf, *rbuf; - pami_data_function pop; - pami_type_t pdt; - int rc; - const int rank = comm_ptr->rank; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - - MPIDI_Datatype_get_info(count, datatype, dt_contig, tsize, dt_null, true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_REDUCE, tsize, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Reduce(sendbuf, recvbuf, count, datatype, op, root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", advisor_algorithms[0].metadata->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - rc = MPIDI_Datatype_to_pami(datatype, &pdt, op, &pop, &mu); - - pami_xfer_t reduce; - const pami_metadata_t *my_reduce_md=NULL; - volatile unsigned reduce_active = 1; - - if(rc != MPI_SUCCESS || !dt_contig) - { - return MPIR_Reduce(sendbuf, recvbuf, count, datatype, op, root, comm_ptr, mpierrno); - } - - - rbuf = (char *)recvbuf + true_lb; - sbuf = (char *)sendbuf + true_lb; - if(sendbuf == MPI_IN_PLACE) - { - sbuf = PAMI_IN_PLACE; - } - - reduce.cb_done = reduce_cb_done; - reduce.cookie = (void *)&reduce_active; - reduce.algorithm = mpid->coll_algorithm[PAMI_XFER_REDUCE][0][0]; - reduce.cmd.xfer_reduce.sndbuf = sbuf; - reduce.cmd.xfer_reduce.rcvbuf = rbuf; - reduce.cmd.xfer_reduce.stype = pdt; - reduce.cmd.xfer_reduce.rtype = pdt; - reduce.cmd.xfer_reduce.stypecount = count; - reduce.cmd.xfer_reduce.rtypecount = count; - reduce.cmd.xfer_reduce.op = pop; - reduce.cmd.xfer_reduce.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - my_reduce_md = &mpid->coll_metadata[PAMI_XFER_REDUCE][0][0]; - - TRACE_ERR("%s reduce, context %d, algoname: %s, exflag: %d\n", MPIDI_Process.context_post.active>0?"Posting":"Invoking", 0, - my_reduce_md->name, exflag); - MPIDI_Post_coll_t reduce_post; - MPIDI_Context_post(MPIDI_Context[0], &reduce_post.state, - MPIDI_Pami_post_wrapper, (void *)&reduce); - TRACE_ERR("Reduce %s\n", MPIDI_Process.context_post.active>0?"posted":"invoked"); - - MPIDI_Update_last_algorithm(comm_ptr, - my_reduce_md->name); - MPID_PROGRESS_WAIT_WHILE(reduce_active); - TRACE_ERR("Reduce done\n"); - return MPI_SUCCESS; -} - - -int -MPIDO_CSWrapper_reduce(pami_xfer_t *reduce, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype type; - MPI_Op op; - void *sbuf; - MPIDI_coll_check_in_place(reduce->cmd.xfer_reduce.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( reduce->cmd.xfer_reduce.stype, - &type, - reduce->cmd.xfer_reduce.op, - &op); - if(rc == -1) return rc; - - - rc = MPIR_Reduce(sbuf, - reduce->cmd.xfer_reduce.rcvbuf, - reduce->cmd.xfer_reduce.rtypecount, type, op, - reduce->cmd.xfer_reduce.root, comm_ptr, &mpierrno); - if(reduce->cb_done && rc == 0) - reduce->cb_done(NULL, reduce->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/scan/Makefile.mk b/src/mpid/pamid/src/coll/scan/Makefile.mk deleted file mode 100644 index 88a44e0..0000000 --- a/src/mpid/pamid/src/coll/scan/Makefile.mk +++ /dev/null @@ -1,29 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/scan/mpido_scan.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/scan/mpido_scan.c b/src/mpid/pamid/src/coll/scan/mpido_scan.c deleted file mode 100644 index ee4c8d2..0000000 --- a/src/mpid/pamid/src/coll/scan/mpido_scan.c +++ /dev/null @@ -1,452 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/gather/mpido_scan.c - * \brief ??? - */ - -/* #define TRACE_ON */ -#include <mpidimpl.h> - -static void scan_cb_done(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_scan enter, active: %u\n", (*active)); - (*active)--; -} -int MPIDO_Doscan(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno, int exflag); - - -int MPIDO_Scan(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno) -{ - return MPIDO_Doscan(sendbuf, recvbuf, count, datatype, - op, comm_ptr, mpierrno, 0); -} - -int MPIDO_Iscan(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, MPIR_Request **request) -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iscan_impl() function to invoke the mpich nbc - * implementation of MPI_Iscan(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iscan() with a blocking MPIR_Scan(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking scan_algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Scan_impl(sendbuf, recvbuf, count, datatype, - op, comm_ptr, &mpierrno); - - /* - * The blocking scan has completed - create and complete a - * MPIR_Request object so the MPIR_Iscan_impl() function - * does not perform an additional iscan. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} - -int MPIDO_Exscan(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno) -{ - return MPIDO_Doscan(sendbuf, recvbuf, count, datatype, - op, comm_ptr, mpierrno, 1); -} - -int MPIDO_Doscan(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno, int exflag) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype*dt_null = NULL; - MPI_Aint true_lb = 0; - int dt_contig ATTRIBUTE((unused)), tsize ATTRIBUTE((unused)); - int mu; - char *sbuf, *rbuf; - pami_data_function pop; - pami_type_t pdt; - int rc; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - int queryreq = 0; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (comm_ptr->rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_SCAN]; - - rc = MPIDI_Datatype_to_pami(datatype, &pdt, op, &pop, &mu); - if(unlikely(verbose)) - fprintf(stderr,"rc %u, dt: %p, op: %p, mu: %u, selectedvar %u != %u (MPICH)\n", - rc, pdt, pop, mu, - (unsigned)selected_type, MPID_COLL_USE_MPICH); - - pami_xfer_t scan; - volatile unsigned scan_active = 1; - - if((selected_type == MPID_COLL_USE_MPICH || rc != MPI_SUCCESS)) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH scan algorithm (exflag %d)\n",exflag); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint dt_extent; - MPIDU_Datatype_get_extent_macro(datatype, dt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = MPIDI_cuda_is_device_buf(sendbuf); - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(dt_extent * count); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, dt_extent * count, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(dt_extent * count); - if(sendbuf == MPI_IN_PLACE) - { - cudaError_t cudaerr = CudaMemcpy(rcbuf, recvbuf, dt_extent * count, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - memset(rcbuf, 0, dt_extent * count); - } - else - rcbuf = recvbuf; - int cuda_res; - if(exflag) - cuda_res = MPIR_Exscan(scbuf, rcbuf, count, datatype, op, comm_ptr, mpierrno); - else - cuda_res = MPIR_Scan(scbuf, rcbuf, count, datatype, op, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, dt_extent * count, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - if(exflag) - return MPIR_Exscan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - else - return MPIR_Scan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - } - - MPIDI_Datatype_get_info(count, datatype, dt_contig, tsize, dt_null, true_lb); - rbuf = (char *)recvbuf + true_lb; - if(sendbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"scan MPI_IN_PLACE buffering\n"); - sbuf = PAMI_IN_PLACE; - } - else - { - sbuf = (char *)sendbuf + true_lb; - } - - scan.cb_done = scan_cb_done; - scan.cookie = (void *)&scan_active; - if(selected_type == MPID_COLL_OPTIMIZED) - { - scan.algorithm = mpid->opt_protocol[PAMI_XFER_SCAN][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_SCAN][0]; - queryreq = mpid->must_query[PAMI_XFER_SCAN][0]; - } - else - { - scan.algorithm = mpid->user_selected[PAMI_XFER_SCAN]; - my_md = &mpid->user_metadata[PAMI_XFER_SCAN]; - queryreq = selected_type; - } - scan.cmd.xfer_scan.sndbuf = sbuf; - scan.cmd.xfer_scan.rcvbuf = rbuf; - scan.cmd.xfer_scan.stype = pdt; - scan.cmd.xfer_scan.rtype = pdt; - scan.cmd.xfer_scan.stypecount = count; - scan.cmd.xfer_scan.rtypecount = count; - scan.cmd.xfer_scan.op = pop; - scan.cmd.xfer_scan.exclusive = exflag; - - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("Querying scan protocol %s, type was %d\n", - my_md->name, - selected_type); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (sendbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb ATTRIBUTE((unused)); - MPIDU_Datatype*data_ptr; - int data_size ATTRIBUTE((unused)), data_contig ATTRIBUTE((unused)); - MPIDI_Datatype_get_info(count, datatype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&scan); - TRACE_ERR("Bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH scan\n", - my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "SCAN_MPICH"); - if(exflag) - return MPIR_Exscan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - else - return MPIR_Scan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for scan on %u (exflag %d)\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id, - exflag); - } - MPIDI_Post_coll_t scan_post; - MPIDI_Context_post(MPIDI_Context[0], &scan_post.state, - MPIDI_Pami_post_wrapper, (void *)&scan); - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - MPID_PROGRESS_WAIT_WHILE(scan_active); - TRACE_ERR("Scan done\n"); - return rc; -} - - -int MPIDO_Doscan_simple(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno, int exflag) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype*dt_null = NULL; - MPI_Aint true_lb = 0; - int dt_contig, tsize; - int mu; - char *sbuf, *rbuf; - pami_data_function pop; - pami_type_t pdt; - int rc; - const pami_metadata_t *my_md; - - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - - pami_xfer_t scan; - volatile unsigned scan_active = 1; - MPIDI_Datatype_get_info(count, datatype, dt_contig, tsize, dt_null, true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_SCAN, tsize, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - if(exflag) - return MPIR_Exscan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - else - return MPIR_Scan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - } - - rc = MPIDI_Datatype_to_pami(datatype, &pdt, op, &pop, &mu); - - if(rc != MPI_SUCCESS || !dt_contig) - { - if(exflag) - return MPIR_Exscan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - else - return MPIR_Scan(sendbuf, recvbuf, count, datatype, op, comm_ptr, mpierrno); - } - - - rbuf = (char *)recvbuf + true_lb; - if(sendbuf == MPI_IN_PLACE) - { - sbuf = PAMI_IN_PLACE; - } - else - { - sbuf = (char *)sendbuf + true_lb; - } - - scan.cb_done = scan_cb_done; - scan.cookie = (void *)&scan_active; - scan.algorithm = mpid->coll_algorithm[PAMI_XFER_SCAN][0][0]; - my_md = &mpid->coll_metadata[PAMI_XFER_SCAN][0][0]; - scan.cmd.xfer_scan.sndbuf = sbuf; - scan.cmd.xfer_scan.rcvbuf = rbuf; - scan.cmd.xfer_scan.stype = pdt; - scan.cmd.xfer_scan.rtype = pdt; - scan.cmd.xfer_scan.stypecount = count; - scan.cmd.xfer_scan.rtypecount = count; - scan.cmd.xfer_scan.op = pop; - scan.cmd.xfer_scan.exclusive = exflag; - - MPIDI_Post_coll_t scan_post; - MPIDI_Context_post(MPIDI_Context[0], &scan_post.state, - MPIDI_Pami_post_wrapper, (void *)&scan); - TRACE_ERR("Scan %s\n", MPIDI_Process.context_post.active>0?"posted":"invoked"); - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - MPID_PROGRESS_WAIT_WHILE(scan_active); - TRACE_ERR("Scan done\n"); - return rc; -} - - -int MPIDO_Exscan_simple(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno) -{ - return MPIDO_Doscan_simple(sendbuf, recvbuf, count, datatype, - op, comm_ptr, mpierrno, 1); -} - -int MPIDO_Scan_simple(const void *sendbuf, void *recvbuf, - int count, MPI_Datatype datatype, - MPI_Op op, MPIR_Comm * comm_ptr, int *mpierrno) -{ - return MPIDO_Doscan_simple(sendbuf, recvbuf, count, datatype, - op, comm_ptr, mpierrno, 0); -} - -int -MPIDO_CSWrapper_scan(pami_xfer_t *scan, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype type; - MPI_Op op; - void *sbuf; - MPIDI_coll_check_in_place(scan->cmd.xfer_scan.sndbuf, &sbuf); - int rc = MPIDI_Dtpami_to_dtmpi( scan->cmd.xfer_scan.stype, - &type, - scan->cmd.xfer_scan.op, - &op); - if(rc == -1) return rc; - - if(scan->cmd.xfer_scan.exclusive) - rc = MPIR_Exscan(sbuf, - scan->cmd.xfer_scan.rcvbuf, - scan->cmd.xfer_scan.rtypecount, type, op, - comm_ptr, &mpierrno); - else - rc = MPIR_Scan(sbuf, - scan->cmd.xfer_scan.rcvbuf, - scan->cmd.xfer_scan.rtypecount, type, op, - comm_ptr, &mpierrno); - if(scan->cb_done && rc == 0) - scan->cb_done(NULL, scan->cookie, PAMI_SUCCESS); - return rc; - -} - diff --git a/src/mpid/pamid/src/coll/scatter/Makefile.mk b/src/mpid/pamid/src/coll/scatter/Makefile.mk deleted file mode 100644 index 9cf6067..0000000 --- a/src/mpid/pamid/src/coll/scatter/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/scatter/mpido_scatter.c \ - src/mpid/pamid/src/coll/scatter/mpido_iscatter.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/scatter/mpido_iscatter.c b/src/mpid/pamid/src/coll/scatter/mpido_iscatter.c deleted file mode 100644 index 9d11d91..0000000 --- a/src/mpid/pamid/src/coll/scatter/mpido_iscatter.c +++ /dev/null @@ -1,74 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/scatter/mpido_iscatter.c - * \brief ??? - */ - -/*#define TRACE_ON */ - -#include <mpidimpl.h> - -int MPIDO_Iscatter(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iscatter_impl() function to invoke the mpich nbc - * implementation of MPI_Iscatter(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iscatter() with a blocking MPIR_Scatter(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking scatter_algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Scatter_impl(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, &mpierrno); - - /* - * The blocking scatter has completed - create and complete a - * MPIR_Request object so the MPIR_Iscatter_impl() function - * does not perform an additional iscatter. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/scatter/mpido_scatter.c b/src/mpid/pamid/src/coll/scatter/mpido_scatter.c deleted file mode 100644 index 038da9f..0000000 --- a/src/mpid/pamid/src/coll/scatter/mpido_scatter.c +++ /dev/null @@ -1,591 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/scatter/mpido_scatter.c - * \brief ??? - */ - -/* #define TRACE_ON */ - -#include <mpidimpl.h> - -static void cb_scatter(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_scatter enter, active: %u\n", (*active)); - MPIDI_Progress_signal(); - (*active)--; -} - -/* works for simple data types, assumes fast bcast is available */ -int MPIDO_Scatter_bcast(void * sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ - /* Pretty simple - bcast a temp buffer and copy our little chunk out */ - int nbytes, rc; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - char *tempbuf = NULL; - - if(rank == root) - { - MPIDI_Datatype_get_data_size(sendcount, - sendtype, - nbytes); - tempbuf = sendbuf; - } - else - { - MPIDI_Datatype_get_data_size(recvcount, - recvtype, - nbytes); - - tempbuf = MPL_malloc(nbytes * size); - if(!tempbuf) - { - return MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, - __FUNCTION__, - __LINE__, - MPI_ERR_OTHER, - "**nomem", 0); - } - } - - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Bcast(tempbuf, nbytes*size, MPI_CHAR, root, comm_ptr, mpierrno); - - if(rank == root && recvbuf == MPI_IN_PLACE) - return rc; - else - memcpy(recvbuf, tempbuf+(rank*nbytes), nbytes); - - if (rank!=root) - MPL_free(tempbuf); - - return rc; -} - - -int MPIDO_Scatter(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype* data_ptr; - MPI_Aint true_lb ATTRIBUTE((unused)); - int contig, nbytes ATTRIBUTE((unused)); - const int rank = comm_ptr->rank; - int success = 1; - pami_type_t stype, rtype; - int tmp; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_SCATTER]; - char use_pami = !(selected_type == MPID_COLL_USE_MPICH); - - /* if (rank == root) - We can't decide on just the root to use MPICH. Really need a pre-allreduce. - For now check sendtype on non-roots too and hope it matches? I think that's what - scatterv does... */ - { - if(MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp) != MPI_SUCCESS) - use_pami = 0; - } - if(recvbuf != MPI_IN_PLACE && (MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp) != MPI_SUCCESS)) - use_pami = 0; - - if(!use_pami) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH scatter algorithm\n"); - MPIDI_Update_last_algorithm(comm_ptr, "SCATTER_MPICH"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = (rank == root) ? MPIDI_cuda_is_device_buf(sendbuf) : 0; - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - scbuf = MPL_malloc(sdt_extent * sendcount); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, sdt_extent * sendcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(rdt_extent * recvcount); - CudaMemcpy(rcbuf, recvbuf, rdt_extent * recvcount, cudaMemcpyDeviceToHost); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Scatter(scbuf, sendcount, sendtype, rcbuf, recvcount, recvtype, root, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, rdt_extent * recvcount, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Scatter(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - - if (rank == root) - { - if (recvtype != MPI_DATATYPE_NULL && recvcount >= 0)/* Should this be send or recv? */ - { - MPIDI_Datatype_get_info(sendcount, sendtype, contig, - nbytes, data_ptr, true_lb); - if (!contig) success = 0; - } - else - success = 0; - - if (success) - { - if (recvtype != MPI_DATATYPE_NULL && recvcount >= 0) - { - MPIDI_Datatype_get_info(recvcount, recvtype, contig, - nbytes, data_ptr, true_lb); - if (!contig) success = 0; - } - else success = 0; - } - } - - else - { - if (sendtype != MPI_DATATYPE_NULL && sendcount >= 0)/* Should this be send or recv? */ - { - MPIDI_Datatype_get_info(recvcount, recvtype, contig, - nbytes, data_ptr, true_lb); - if (!contig) success = 0; - } - else - success = 0; - } - - - pami_xfer_t scatter; - MPIDI_Post_coll_t scatter_post; - pami_algorithm_t my_scatter; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - volatile unsigned scatter_active = 1; - int queryreq = 0; - - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized scatter %s was selected\n", - mpid->opt_protocol_md[PAMI_XFER_SCATTER][0].name); - my_scatter = mpid->opt_protocol[PAMI_XFER_SCATTER][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_SCATTER][0]; - queryreq = mpid->must_query[PAMI_XFER_SCATTER][0]; - } - else - { - TRACE_ERR("Optimized scatter %s was set by user\n", - mpid->user_metadata[PAMI_XFER_SCATTER].name); - my_scatter = mpid->user_selected[PAMI_XFER_SCATTER]; - my_md = &mpid->user_metadata[PAMI_XFER_SCATTER]; - queryreq = selected_type; - } - - scatter.algorithm = my_scatter; - scatter.cb_done = cb_scatter; - scatter.cookie = (void *)&scatter_active; - scatter.cmd.xfer_scatter.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - scatter.cmd.xfer_scatter.sndbuf = (void *)sendbuf; - scatter.cmd.xfer_scatter.stype = stype; - scatter.cmd.xfer_scatter.stypecount = sendcount; - if(recvbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"scatter MPI_IN_PLACE buffering\n"); - MPIDI_Datatype_get_info(sendcount, sendtype, contig, - nbytes, data_ptr, true_lb); - scatter.cmd.xfer_scatter.rcvbuf = PAMI_IN_PLACE; - scatter.cmd.xfer_scatter.rtype = stype; - scatter.cmd.xfer_scatter.rtypecount = sendcount; - } - else - { - scatter.cmd.xfer_scatter.rcvbuf = (void *)recvbuf; - scatter.cmd.xfer_scatter.rtype = rtype; - scatter.cmd.xfer_scatter.rtypecount = recvcount; - } - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying scatter protoocl %s, type was %d\n", - my_md->name, queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (recvbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; - if(my_md->check_correct.values.rangeminmax) - { - int data_size; - if(rank == root) - MPIDI_Datatype_get_data_size(sendcount, sendtype, data_size); - else - MPIDI_Datatype_get_data_size(recvcount, recvtype, data_size); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; /* ok, algorithm selected */ - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&scatter); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"query failed for %s\n", my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "SCATTER_MPICH"); - return MPIR_Scatter(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for scatter on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - MPIDI_Context_post(MPIDI_Context[0], &scatter_post.state, - MPIDI_Pami_post_wrapper, (void *)&scatter); - TRACE_ERR("Waiting on active %d\n", scatter_active); - MPID_PROGRESS_WAIT_WHILE(scatter_active); - - - TRACE_ERR("Leaving MPIDO_Scatter\n"); - - return 0; -} - - -int MPIDO_Scatter_simple(const void *sendbuf, - int sendcount, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - MPIDU_Datatype* data_ptr; - const int rank = comm_ptr->rank; - int success = 1, snd_contig = 1, rcv_contig = 1; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - size_t send_size = 0, recv_size = 0, data_size = 0; - MPI_Aint snd_true_lb = 0, rcv_true_lb = 0; - MPIDU_Segment segment; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int size = comm_ptr->local_size; - advisor_algorithm_t advisor_algorithms[1]; - - if (rank == root) - { - MPIDI_Datatype_get_info(sendcount, sendtype, snd_contig, - send_size, data_ptr, snd_true_lb); - - if (recvbuf != MPI_IN_PLACE && recvtype != MPI_DATATYPE_NULL && recvcount >= 0) - { - MPIDI_Datatype_get_info(recvcount, recvtype, rcv_contig, - recv_size, data_ptr, rcv_true_lb); - } - data_size = send_size; - } - else if (recvtype != MPI_DATATYPE_NULL && recvcount >= 0) - { - MPIDI_Datatype_get_info(recvcount, recvtype, rcv_contig, - recv_size, data_ptr, rcv_true_lb); - data_size = recv_size; - } - - advisor_algorithms[0].metadata = NULL;/* We check for NULL further down */ - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, - PAMI_XFER_SCATTER, data_size, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Scatter(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - } - } - sbuf = (char *)sendbuf + snd_true_lb; - rbuf = (char *)recvbuf + rcv_true_lb; - if (rank == root) - { - if (send_size) - { - if (!snd_contig) - { - snd_noncontig_buff = MPL_malloc(send_size * size); - sbuf = snd_noncontig_buff; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - DLOOP_Offset last = send_size * size; - MPIDU_Segment_init(sendbuf, sendcount * size, sendtype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, snd_noncontig_buff); - } - } - else - success = 0; - - if (success && recvbuf != MPI_IN_PLACE) - { - if (recv_size) - { - if (!rcv_contig) - { - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - } - } - else success = 0; - } - } - else - { - if (recv_size)/* Should this be send or recv? */ - { - if (!rcv_contig) - { - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - } - } - else - success = 0; - } - - if(!success) - { - MPIDI_Update_last_algorithm(comm_ptr, "SCATTER_MPICH"); - return MPIR_Scatter(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - - if(advisor_algorithms[0].metadata && - advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && - !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - - pami_xfer_t scatter; - MPIDI_Post_coll_t scatter_post; - volatile unsigned scatter_active = 1; - - - scatter.algorithm = mpid->coll_algorithm[PAMI_XFER_SCATTER][0][0]; - - scatter.cb_done = cb_scatter; - scatter.cookie = (void *)&scatter_active; - scatter.cmd.xfer_scatter.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - scatter.cmd.xfer_scatter.sndbuf = (void *)sbuf; - scatter.cmd.xfer_scatter.stype = PAMI_TYPE_BYTE; - scatter.cmd.xfer_scatter.stypecount = send_size; - scatter.cmd.xfer_scatter.rcvbuf = (void *)rbuf; - scatter.cmd.xfer_scatter.rtype = PAMI_TYPE_BYTE;/* rtype is ignored when rcvbuf == PAMI_IN_PLACE */ - scatter.cmd.xfer_scatter.rtypecount = recv_size; - - if(recvbuf == MPI_IN_PLACE) - { - scatter.cmd.xfer_scatter.rcvbuf = PAMI_IN_PLACE; - } - - TRACE_ERR("%s scatter\n", MPIDI_Process.context_post.active>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &scatter_post.state, - MPIDI_Pami_post_wrapper, (void *)&scatter); - TRACE_ERR("Waiting on active %d\n", scatter_active); - MPID_PROGRESS_WAIT_WHILE(scatter_active); - - if(!rcv_contig && rcv_noncontig_buff) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size, MPI_CHAR, - recvbuf, recvcount, recvtype); - MPL_free(rcv_noncontig_buff); - } - if(!snd_contig) MPL_free(snd_noncontig_buff); - - - TRACE_ERR("Leaving MPIDO_Scatter_optimized\n"); - - return MPI_SUCCESS; -} - - -int -MPIDO_CSWrapper_scatter(pami_xfer_t *scatter, - void *comm) -{ - int mpierrno = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *rbuf; - MPIDI_coll_check_in_place(scatter->cmd.xfer_scatter.rcvbuf, &rbuf); - int rc = MPIDI_Dtpami_to_dtmpi( scatter->cmd.xfer_scatter.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( scatter->cmd.xfer_scatter.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Scatter(scatter->cmd.xfer_scatter.sndbuf, - scatter->cmd.xfer_scatter.stypecount, sendtype, - rbuf, - scatter->cmd.xfer_scatter.rtypecount, recvtype, - scatter->cmd.xfer_scatter.root, comm_ptr, &mpierrno); - if(scatter->cb_done && rc == 0) - scatter->cb_done(NULL, scatter->cookie, PAMI_SUCCESS); - return rc; - -} - -#if 0 /* old glue-based scatter-via-bcast */ - - /* Assume glue protocol sucks for now.... Needs work if not or to enable */ - - - - /* see if we all agree to use bcast scatter */ - MPIDO_Allreduce(MPI_IN_PLACE, &success, 1, MPI_INT, MPI_BAND, comm_ptr, mpierrno); - - if (!success) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH scatter algorithm\n"); - return MPIR_Scatter(sendbuf, sendcount, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - - sbuf = (char *)sendbuf+true_lb; - rbuf = (char *)recvbuf+true_lb; - - MPIDI_Update_last_algorithm(comm_ptr, "SCATTER_OPT_BCAST"); - return MPIDO_Scatter_bcast(sbuf, sendcount, sendtype, - rbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); -#endif diff --git a/src/mpid/pamid/src/coll/scatterv/Makefile.mk b/src/mpid/pamid/src/coll/scatterv/Makefile.mk deleted file mode 100644 index 7e00f99..0000000 --- a/src/mpid/pamid/src/coll/scatterv/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/coll/scatterv/mpido_scatterv.c \ - src/mpid/pamid/src/coll/scatterv/mpido_iscatterv.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/coll/scatterv/mpido_iscatterv.c b/src/mpid/pamid/src/coll/scatterv/mpido_iscatterv.c deleted file mode 100644 index 11d9173..0000000 --- a/src/mpid/pamid/src/coll/scatterv/mpido_iscatterv.c +++ /dev/null @@ -1,73 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/scatterv/mpido_iscatterv.c - * \brief ??? - */ - -#include <mpidimpl.h> - -int MPIDO_Iscatterv(const void *sendbuf, - const int *sendcounts, - const int *displs, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - MPIR_Request **request) -{ - /*if (unlikely((data_size == 0) || (user_selected_type == MPID_COLL_USE_MPICH)))*/ - { - /* - * If the mpich mpir non-blocking collectives are enabled, return without - * first constructing the MPIR_Request. This signals to the - * MPIR_Iscatterv_impl() function to invoke the mpich nbc - * implementation of MPI_Iscatterv(). - */ - if (MPIDI_Process.mpir_nbc != 0) - return 0; - - /* - * MPIR_* nbc implementation is not enabled. Fake a non-blocking - * MPIR_Iscatterv() with a blocking MPIR_Scatterv(). - */ - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && comm_ptr->rank == 0)) - fprintf(stderr,"Using MPICH blocking scatterv_algorithm\n"); - - int mpierrno = 0; - int rc = MPIR_Scatterv_impl(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, &mpierrno); - - /* - * The blocking scatterv has completed - create and complete a - * MPIR_Request object so the MPIR_Iscatterv_impl() function - * does not perform an additional iscatterv. - */ - MPIR_Request * mpid_request = MPID_Request_create_inline(); - mpid_request->kind = MPIR_REQUEST_KIND__COLL; - *request = mpid_request; - MPIDI_Request_complete_norelease_inline(mpid_request); - - return rc; - } - - return 0; -} diff --git a/src/mpid/pamid/src/coll/scatterv/mpido_scatterv.c b/src/mpid/pamid/src/coll/scatterv/mpido_scatterv.c deleted file mode 100644 index 75c5186..0000000 --- a/src/mpid/pamid/src/coll/scatterv/mpido_scatterv.c +++ /dev/null @@ -1,812 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/coll/scatterv/mpido_scatterv.c - * \brief ??? - */ - -#include <mpidimpl.h> - - -/* basically, everyone receives recvcount via bcast */ -/* requires a contiguous/continous buffer on root though */ -int MPIDO_Scatterv_bcast(void *sendbuf, - int *sendcounts, - int *displs, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - char *tempbuf; - int i, sum = 0, dtsize, rc=0; - - for (i = 0; i < size; i++) - if (sendcounts > 0) - sum += sendcounts[i]; - - MPIDI_Datatype_get_data_size(1, - recvtype, - dtsize); - - if (rank != root) - { - tempbuf = MPL_malloc(dtsize * sum); - if (!tempbuf) - return MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, - __FUNCTION__, - __LINE__, - MPI_ERR_OTHER, - "**nomem", 0); - } - else - tempbuf = sendbuf; - - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Bcast(tempbuf, sum, sendtype, root, comm_ptr, mpierrno); - - if(rank == root && recvbuf == MPI_IN_PLACE) - return rc; - - memcpy(recvbuf, tempbuf + displs[rank], sendcounts[rank] * dtsize); - - if (rank != root) - MPL_free(tempbuf); - - return rc; -} - -/* this guy requires quite a few buffers. maybe - * we should somehow "steal" the comm_ptr alltoall ones? */ -int MPIDO_Scatterv_alltoallv(void * sendbuf, - int * sendcounts, - int * displs, - MPI_Datatype sendtype, - void * recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm * comm_ptr, - int *mpierrno) -{ - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - - int *sdispls, *scounts; - int *rdispls, *rcounts; - char *sbuf, *rbuf; - int rbytes; - int rc; - - MPIDI_Datatype_get_data_size(recvcount, - recvtype, - rbytes); - - rbuf = MPL_malloc(size * rbytes * sizeof(char)); - if(!rbuf) - { - return MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, - __FUNCTION__, - __LINE__, - MPI_ERR_OTHER, - "**nomem", 0); - } - - if(rank == root) - { - sdispls = displs; - scounts = sendcounts; - sbuf = sendbuf; - } - else - { - sdispls = MPL_malloc(size * sizeof(int)); - scounts = MPL_malloc(size * sizeof(int)); - sbuf = MPL_malloc(rbytes * sizeof(char)); - if(!sdispls || !scounts || !sbuf) - { - if(sdispls) - MPL_free(sdispls); - if(scounts) - MPL_free(scounts); - return MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, - __FUNCTION__, - __LINE__, - MPI_ERR_OTHER, - "**nomem", 0); - } - memset(sdispls, 0, size*sizeof(int)); - memset(scounts, 0, size*sizeof(int)); - } - - rdispls = MPL_malloc(size * sizeof(int)); - rcounts = MPL_malloc(size * sizeof(int)); - if(!rdispls || !rcounts) - { - if(rdispls) - MPL_free(rdispls); - return MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, - __FUNCTION__, - __LINE__, - MPI_ERR_OTHER, - "**nomem", 0); - } - - memset(rdispls, 0, size*sizeof(unsigned)); - memset(rcounts, 0, size*sizeof(unsigned)); - - rcounts[root] = rbytes; - - /* Switch to comm->coll_fns->fn() */ - rc = MPIDO_Alltoallv(sbuf, - scounts, - sdispls, - sendtype, - rbuf, - rcounts, - rdispls, - MPI_CHAR, - comm_ptr, - mpierrno); - - if(rank == root && recvbuf == MPI_IN_PLACE) - { - MPL_free(rbuf); - MPL_free(rdispls); - MPL_free(rcounts); - return rc; - } - else - { - memcpy(recvbuf, rbuf, rbytes); - MPL_free(rbuf); - MPL_free(rdispls); - MPL_free(rcounts); - if(rank != root) - { - MPL_free(sbuf); - MPL_free(sdispls); - MPL_free(scounts); - } - } - - return rc; -} - -static void cb_scatterv(void *ctxt, void *clientdata, pami_result_t err) -{ - unsigned *active = (unsigned *)clientdata; - TRACE_ERR("cb_scatterv enter, active: %u\n", (*active)); - (*active)--; -} - -int MPIDO_Scatterv(const void *sendbuf, - const int *sendcounts, - const int *displs, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - int tmp, i, pamidt = 1; - int contig ATTRIBUTE((unused)); - int ssize ATTRIBUTE((unused)); - int rsize ATTRIBUTE((unused)); - MPIDU_Datatype*dt_ptr = NULL; - MPI_Aint send_true_lb=0, recv_true_lb; - char *sbuf, *rbuf; - pami_type_t stype, rtype; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; -#if ASSERT_LEVEL==0 - /* We can't afford the tracing in ndebug/performance libraries */ - const unsigned verbose = 0; -#else - const unsigned verbose = (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) && (rank == 0); -#endif - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - const int selected_type = mpid->user_selected_type[PAMI_XFER_SCATTERV_INT]; - - if(selected_type == MPID_COLL_USE_MPICH) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH scatterv algorithm\n"); - MPIDI_Update_last_algorithm(comm_ptr, "SCATTERV_MPICH"); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - MPI_Aint sdt_extent,rdt_extent; - MPIDU_Datatype_get_extent_macro(sendtype, sdt_extent); - MPIDU_Datatype_get_extent_macro(recvtype, rdt_extent); - char *scbuf = NULL; - char *rcbuf = NULL; - int is_send_dev_buf = (rank == root) ? MPIDI_cuda_is_device_buf(sendbuf) : 0; - int is_recv_dev_buf = MPIDI_cuda_is_device_buf(recvbuf); - if(is_send_dev_buf) - { - //Since displs can be non-continous, we need to calculate max buffer size - int highest_displs = displs[size - 1]; - int highest_sendcount = sendcounts[size - 1]; - size_t stotal_buf; - for(i = 0; i < size; i++) - { - if(displs[i]+sendcounts[i] > highest_displs+highest_sendcount) - { - highest_displs = displs[i]; - highest_sendcount = sendcounts[i]; - } - } - stotal_buf = (highest_displs+highest_sendcount)*sdt_extent; - scbuf = MPL_malloc(stotal_buf); - cudaError_t cudaerr = CudaMemcpy(scbuf, sendbuf, stotal_buf, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - } - else - scbuf = sendbuf; - if(is_recv_dev_buf) - { - rcbuf = MPL_malloc(recvcount * rdt_extent); - CudaMemcpy(rcbuf, recvbuf, recvcount * rdt_extent, cudaMemcpyDeviceToHost); - } - else - rcbuf = recvbuf; - int cuda_res = MPIR_Scatterv(scbuf, sendcounts, displs, sendtype, rcbuf, recvcount, recvtype, root, comm_ptr, mpierrno); - if(is_send_dev_buf)MPL_free(scbuf); - if(is_recv_dev_buf) - { - cudaError_t cudaerr = CudaMemcpy(recvbuf, rcbuf, recvcount * rdt_extent, cudaMemcpyHostToDevice); - if (cudaSuccess != cudaerr) - fprintf(stderr, "cudaMemcpy failed: %s\n", CudaGetErrorString(cudaerr)); - MPL_free(rcbuf); - } - return cuda_res; - } - else -#endif - return MPIR_Scatterv(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - - - pami_xfer_t scatterv; - pami_algorithm_t my_scatterv; - const pami_metadata_t *my_md = (pami_metadata_t *)NULL; - volatile unsigned scatterv_active = 1; - int queryreq = 0; - - if(selected_type == MPID_COLL_OPTIMIZED) - { - TRACE_ERR("Optimized scatterv %s was selected\n", - mpid->opt_protocol_md[PAMI_XFER_SCATTERV_INT][0].name); - my_scatterv = mpid->opt_protocol[PAMI_XFER_SCATTERV_INT][0]; - my_md = &mpid->opt_protocol_md[PAMI_XFER_SCATTERV_INT][0]; - queryreq = mpid->must_query[PAMI_XFER_SCATTERV_INT][0]; - } - else - { - TRACE_ERR("User selected %s for scatterv\n", - mpid->user_selected[PAMI_XFER_SCATTERV_INT]); - my_scatterv = mpid->user_selected[PAMI_XFER_SCATTERV_INT]; - my_md = &mpid->user_metadata[PAMI_XFER_SCATTERV_INT]; - queryreq = selected_type; - } - - if((recvbuf != MPI_IN_PLACE) && MPIDI_Datatype_to_pami(recvtype, &rtype, -1, NULL, &tmp) != MPI_SUCCESS) - pamidt = 0; - - if(MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp) != MPI_SUCCESS) - pamidt = 0; - - if(pamidt == 0 || selected_type == MPID_COLL_USE_MPICH) - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH scatterv algorithm\n"); - TRACE_ERR("Scatterv using MPICH\n"); - MPIDI_Update_last_algorithm(comm_ptr, "SCATTERV_MPICH"); - return MPIR_Scatterv(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - - MPIDI_Datatype_get_info(1, sendtype, contig, ssize, dt_ptr, send_true_lb); - sbuf = (char *)sendbuf + send_true_lb; - rbuf = recvbuf; - - if(rank == root) - { - if(recvbuf == MPI_IN_PLACE) - { - if(unlikely(verbose)) - fprintf(stderr,"scatterv MPI_IN_PLACE buffering\n"); - rbuf = PAMI_IN_PLACE; - } - else - { - MPIDI_Datatype_get_info(1, recvtype, contig, rsize, dt_ptr, recv_true_lb); - rbuf = (char *)recvbuf + recv_true_lb; - } - } - - scatterv.cb_done = cb_scatterv; - scatterv.cookie = (void *)&scatterv_active; - scatterv.cmd.xfer_scatterv_int.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - - scatterv.algorithm = my_scatterv; - - scatterv.cmd.xfer_scatterv_int.rcvbuf = rbuf; - scatterv.cmd.xfer_scatterv_int.sndbuf = sbuf; - scatterv.cmd.xfer_scatterv_int.stype = stype; - scatterv.cmd.xfer_scatterv_int.rtype = rtype; - scatterv.cmd.xfer_scatterv_int.stypecounts = (int *) sendcounts; - scatterv.cmd.xfer_scatterv_int.rtypecount = recvcount; - scatterv.cmd.xfer_scatterv_int.sdispls = (int *) displs; - - if(unlikely(queryreq == MPID_COLL_ALWAYS_QUERY || - queryreq == MPID_COLL_CHECK_FN_REQUIRED)) - { - metadata_result_t result = {0}; - TRACE_ERR("querying scatterv protocol %s, type was %d\n", - my_md->name, queryreq); - if(my_md->check_fn == NULL) - { - /* process metadata bits */ - if((!my_md->check_correct.values.inplace) && (recvbuf == MPI_IN_PLACE)) - result.check.unspecified = 1; -/* Can't check ranges like this. Non-local. Comment out for now. - if(my_md->check_correct.values.rangeminmax) - { - MPI_Aint data_true_lb; - MPIDU_Datatype*data_ptr; - int data_size, data_contig; - MPIDI_Datatype_get_info(recvcount, recvtype, data_contig, data_size, data_ptr, data_true_lb); - if((my_md->range_lo <= data_size) && - (my_md->range_hi >= data_size)) - ; - else - { - result.check.range = 1; - if(unlikely(verbose)) - { - fprintf(stderr,"message size (%u) outside range (%zu<->%zu) for %s.\n", - data_size, - my_md->range_lo, - my_md->range_hi, - my_md->name); - } - } - } - */ - } - else /* calling the check fn is sufficient */ - result = my_md->check_fn(&scatterv); - TRACE_ERR("bitmask: %#X\n", result.bitmask); - result.check.nonlocal = 0; /* #warning REMOVE THIS WHEN IMPLEMENTED */ - if(result.bitmask) - { - if(unlikely(verbose)) - fprintf(stderr,"Query failed for %s. Using MPICH scatterv.\n", my_md->name); - MPIDI_Update_last_algorithm(comm_ptr, "SCATTERV_MPICH"); - return MPIR_Scatterv(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - if(my_md->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - if(unlikely(verbose)) - fprintf(stderr,"Query barrier required for %s\n", my_md->name); - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - } - - MPIDI_Update_last_algorithm(comm_ptr, my_md->name); - - if(unlikely(verbose)) - { - unsigned long long int threadID; - MPL_thread_id_t tid; - MPL_thread_self(&tid); - threadID = (unsigned long long int)tid; - fprintf(stderr,"<%llx> Using protocol %s for scatterv on %u\n", - threadID, - my_md->name, - (unsigned) comm_ptr->context_id); - } - MPIDI_Post_coll_t scatterv_post; - MPIDI_Context_post(MPIDI_Context[0], &scatterv_post.state, - MPIDI_Pami_post_wrapper, (void *)&scatterv); - - TRACE_ERR("Waiting on active %d\n", scatterv_active); - MPID_PROGRESS_WAIT_WHILE(scatterv_active); - - TRACE_ERR("Leaving MPIDO_Scatterv\n"); - return 0; -} - -int MPIDO_Scatterv_simple(const void *sendbuf, - const int *sendcounts, - const int *displs, - MPI_Datatype sendtype, - void *recvbuf, - int recvcount, - MPI_Datatype recvtype, - int root, - MPIR_Comm *comm_ptr, - int *mpierrno) -{ -#ifndef HAVE_PAMI_IN_PLACE - if (sendbuf == MPI_IN_PLACE) - { - MPID_Abort (NULL, 0, 1, "'MPI_IN_PLACE' requries support for `PAMI_IN_PLACE`"); - return -1; - } -#endif - int snd_contig = 1; - int rcv_contig = 1; - int send_size = 0, recv_size = 0; - int ssize = 0; - MPIDU_Datatype*dt_ptr = NULL; - MPI_Aint send_true_lb=0, recv_true_lb=0; - void *snd_noncontig_buff = NULL, *rcv_noncontig_buff = NULL; - void *sbuf = NULL, *rbuf = NULL; - int *sdispls = NULL, *scounts = NULL; - int sndcount = 0; - MPIDU_Segment segment; - int tmp, i; - pami_type_t stype = PAMI_TYPE_NULL; - const int rank = comm_ptr->rank; - const int size = comm_ptr->local_size; - const struct MPIDI_Comm* const mpid = &(comm_ptr->mpid); - - if (rank == root && sendtype != MPI_DATATYPE_NULL && sendcounts[0] >= 0) - { - MPIDI_Datatype_get_info(1, sendtype, snd_contig, ssize, dt_ptr, send_true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_SCATTERV_INT, 64, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Scatterv(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - - } - } - } - - if (recvtype != MPI_DATATYPE_NULL && recvcount >= 0) - { - MPIDI_Datatype_get_info(recvcount, recvtype, rcv_contig, - recv_size, dt_ptr, recv_true_lb); - if(MPIDI_Pamix_collsel_advise != NULL && mpid->collsel_fast_query != NULL) - { - advisor_algorithm_t advisor_algorithms[1]; - int num_algorithms = MPIDI_Pamix_collsel_advise(mpid->collsel_fast_query, PAMI_XFER_SCATTERV_INT, 64, advisor_algorithms, 1); - if(num_algorithms) - { - if(advisor_algorithms[0].algorithm_type == COLLSEL_EXTERNAL_ALGO) - { - return MPIR_Scatterv(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } - else if(advisor_algorithms[0].metadata && advisor_algorithms[0].metadata->check_correct.values.asyncflowctl && !(--(comm_ptr->mpid.num_requests))) - { - comm_ptr->mpid.num_requests = MPIDI_Process.optimized.num_requests; - int tmpmpierrno; - MPIDO_Barrier(comm_ptr, &tmpmpierrno); - } - - } - } - } - - pami_xfer_t scatterv; - const pami_metadata_t *my_scatterv_md; - volatile unsigned scatterv_active = 1; - - sbuf = (char *)sendbuf + send_true_lb; - rbuf = (char *)recvbuf + recv_true_lb; - scounts = (int*)sendcounts; - sdispls = (int*)displs; - if(rank == root) - { - if(MPIDI_Datatype_to_pami(sendtype, &stype, -1, NULL, &tmp) != MPI_SUCCESS) - { - if (!snd_contig) - { - scounts = (int*)MPL_malloc(size); - sdispls = (int*)MPL_malloc(size); - for(i = 0; i < size; i++) - { - scounts[i] = ssize * sendcounts[i]; - sdispls[i] = ssize * displs[i]; - send_size += scounts[i]; - sndcount += sendcounts[i]; - } - snd_noncontig_buff = MPL_malloc(send_size); - sbuf = snd_noncontig_buff; - stype = PAMI_TYPE_BYTE; - if(snd_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - DLOOP_Offset last = send_size; - MPIDU_Segment_init(sendbuf, sndcount, sendtype, &segment, 0); - MPIDU_Segment_pack(&segment, 0, &last, snd_noncontig_buff); - } - } - if(recvbuf == MPI_IN_PLACE) - { - rbuf = PAMI_IN_PLACE; - } - } - - if(recvbuf != MPI_IN_PLACE) - { - if (!rcv_contig) - { - rcv_noncontig_buff = MPL_malloc(recv_size); - rbuf = rcv_noncontig_buff; - if(rcv_noncontig_buff == NULL) - { - MPID_Abort(NULL, MPI_ERR_NO_SPACE, 1, - "Fatal: Cannot allocate pack buffer"); - } - } - } - - scatterv.cb_done = cb_scatterv; - scatterv.cookie = (void *)&scatterv_active; - scatterv.cmd.xfer_scatterv_int.root = MPIDI_Task_to_endpoint(MPID_VCR_GET_LPID(comm_ptr->vcr, root), 0); - - scatterv.algorithm = mpid->coll_algorithm[PAMI_XFER_SCATTERV_INT][0][0]; - my_scatterv_md = &mpid->coll_metadata[PAMI_XFER_SCATTERV_INT][0][0]; - - scatterv.cmd.xfer_scatterv_int.rcvbuf = rbuf; - scatterv.cmd.xfer_scatterv_int.sndbuf = sbuf; - scatterv.cmd.xfer_scatterv_int.stype = stype; - scatterv.cmd.xfer_scatterv_int.rtype = PAMI_TYPE_BYTE;/* rtype is ignored when rcvbuf == PAMI_IN_PLACE */ - scatterv.cmd.xfer_scatterv_int.stypecounts = (int *) scounts; - scatterv.cmd.xfer_scatterv_int.rtypecount = recv_size; - scatterv.cmd.xfer_scatterv_int.sdispls = (int *) sdispls; - - - MPIDI_Update_last_algorithm(comm_ptr, my_scatterv_md->name); - - - MPIDI_Post_coll_t scatterv_post; - TRACE_ERR("%s scatterv\n", MPIDI_Process.context_post.active>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &scatterv_post.state, - MPIDI_Pami_post_wrapper, (void *)&scatterv); - - TRACE_ERR("Waiting on active %d\n", scatterv_active); - MPID_PROGRESS_WAIT_WHILE(scatterv_active); - - if(!rcv_contig) - { - MPIR_Localcopy(rcv_noncontig_buff, recv_size, MPI_CHAR, - recvbuf, recvcount, recvtype); - MPL_free(rcv_noncontig_buff); - } - if(!snd_contig) - { - MPL_free(snd_noncontig_buff); - MPL_free(scounts); - MPL_free(sdispls); - } - - TRACE_ERR("Leaving MPIDO_Scatterv_optimized\n"); - return MPI_SUCCESS; -} - - -int -MPIDO_CSWrapper_scatterv(pami_xfer_t *scatterv, - void *comm) -{ - int mpierrno = 0, rc = 0; - MPIR_Comm *comm_ptr = (MPIR_Comm*)comm; - MPI_Datatype sendtype, recvtype; - void *rbuf; - MPIDI_coll_check_in_place(scatterv->cmd.xfer_scatterv_int.rcvbuf, &rbuf); - /* Since collective selection in PAMI fixes message size when selecting the - algorithm, this wrapper function may be called back from PAMI */ - if(PAMI_TYPE_NULL == scatterv->cmd.xfer_scatterv_int.stype) - sendtype = MPI_DATATYPE_NULL; - else - rc = MPIDI_Dtpami_to_dtmpi( scatterv->cmd.xfer_scatterv_int.stype, - &sendtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIDI_Dtpami_to_dtmpi( scatterv->cmd.xfer_scatterv_int.rtype, - &recvtype, - NULL, - NULL); - if(rc == -1) return rc; - - rc = MPIR_Scatterv(scatterv->cmd.xfer_scatterv_int.sndbuf, - scatterv->cmd.xfer_scatterv_int.stypecounts, - scatterv->cmd.xfer_scatterv_int.sdispls, sendtype, - rbuf, - scatterv->cmd.xfer_scatterv_int.rtypecount, recvtype, - scatterv->cmd.xfer_scatterv_int.root, comm_ptr, &mpierrno); - if(scatterv->cb_done && rc == 0) - scatterv->cb_done(NULL, scatterv->cookie, PAMI_SUCCESS); - return rc; - -} - - -#if 0 -/* remove the glue-based optimized scattervs for now. */ - - - /* we can't call scatterv-via-bcast unless we know all nodes have - * valid sendcount arrays. so the user must explicitly ask for it. - */ - - /* optscatterv[0] == optscatterv bcast? - * optscatterv[1] == optscatterv alltoall? - * (having both allows cutoff agreement) - * optscatterv[2] == sum of sendcounts - */ - - optscatterv[0] = !mpid->scattervs[0]; - optscatterv[1] = !mpid->scattervs[1]; - optscatterv[2] = 1; - - if(rank == root) - { - if(!optscatterv[1]) - { - if(sendcounts) - { - for(i=0;i<size;i++) - sum+=sendcounts[i]; - } - optscatterv[2] = sum; - } - - MPIDI_Datatype_get_info(1, - sendtype, - contig, - nbytes, - dt_ptr, - true_lb); - if(recvtype == MPI_DATATYPE_NULL || recvcount <= 0 || !contig) - { - optscatterv[0] = 1; - optscatterv[1] = 1; - } - } - else - { - MPIDI_Datatype_get_info(1, - recvtype, - contig, - nbytes, - dt_ptr, - true_lb); - if(sendtype == MPI_DATATYPE_NULL || !contig) - { - optscatterv[0] = 1; - optscatterv[1] = 1; - } - } - - /* Make sure parameters are the same on all the nodes */ - /* specifically, noncontig on the receive */ - /* set the internal control flow to disable internal star tuning */ - if(mpid->preallreduces[MPID_SCATTERV_PREALLREDUCE]) - { - MPIDI_Post_coll_t allred_post; - rc = MPIDI_Context_post(MPIDI_Context[0], &allred_post.state, - MPIDI_Pami_post_wrapper, (void *)&allred); - - MPID_PROGRESS_WAIT_WHILE(allred_active); - } - /* reset flag */ - - if(!optscatterv[0] || (!optscatterv[1])) - { - char *newrecvbuf = recvbuf; - char *newsendbuf = sendbuf; - if(rank == root) - { - newsendbuf = (char *)sendbuf + true_lb; - } - else - { - newrecvbuf = (char *)recvbuf + true_lb; - } - if(!optscatterv[0]) - { - MPIDI_Update_last_algorithm(comm_ptr, "SCATTERV_OPT_ALLTOALL"); - return MPIDO_Scatterv_alltoallv(newsendbuf, - sendcounts, - displs, - sendtype, - newrecvbuf, - recvcount, - recvtype, - root, - comm_ptr, - mpierrno); - - } - else - { - MPIDI_Update_last_algorithm(comm_ptr, "SCATTERV_OPT_BCAST"); - return MPIDO_Scatterv_bcast(newsendbuf, - sendcounts, - displs, - sendtype, - newrecvbuf, - recvcount, - recvtype, - root, - comm_ptr, - mpierrno); - } - } /* nothing valid to try, go to mpich */ - else - { - if(unlikely(verbose)) - fprintf(stderr,"Using MPICH scatterv algorithm\n"); - MPIDI_Update_last_algorithm(comm_ptr, "SCATTERV_MPICH"); - return MPIR_Scatterv(sendbuf, sendcounts, displs, sendtype, - recvbuf, recvcount, recvtype, - root, comm_ptr, mpierrno); - } -#endif diff --git a/src/mpid/pamid/src/comm/Makefile.mk b/src/mpid/pamid/src/comm/Makefile.mk deleted file mode 100644 index de786c7..0000000 --- a/src/mpid/pamid/src/comm/Makefile.mk +++ /dev/null @@ -1,31 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/comm/mpid_comm.c \ - src/mpid/pamid/src/comm/mpid_selectcolls.c \ - src/mpid/pamid/src/comm/mpid_optcolls.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/comm/mpid_comm.c b/src/mpid/pamid/src/comm/mpid_comm.c deleted file mode 100644 index 34bb908..0000000 --- a/src/mpid/pamid/src/comm/mpid_comm.c +++ /dev/null @@ -1,432 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/comm/mpid_comm.c - * \brief ??? - */ - -/*#define TRACE_ON */ - -#include <mpidimpl.h> -extern void MPIDI_Comm_coll_query(MPIR_Comm *); -extern void MPIDI_Comm_coll_envvars(MPIR_Comm *); - -void geom_create_cb_done(void *ctxt, void *data, pami_result_t err) -{ - int *active = (int *)data; - (*active)--; -} - -void geom_destroy_cb_done(void *ctxt, void *data, pami_result_t err) -{ - int *active = (int *)data; - (*active)--; -} - -int MPIDI_Comm_create (MPIR_Comm *comm) -{ - MPIDI_Coll_comm_create(comm); - return MPI_SUCCESS; -} - -int MPIDI_Comm_destroy (MPIR_Comm *comm) -{ - MPIDI_Coll_comm_destroy(comm); - return MPI_SUCCESS; -} - -pami_result_t MPIDI_Comm_create_from_pami_geom(pami_geometry_range_t *task_slices, - size_t slice_count, - pami_geometry_t *geometry, - void **cookie) -{ - int mpi_errno = MPI_SUCCESS; - int num_tasks = 0; - int *ranks = NULL; - MPIR_Comm *comm_ptr = NULL, *new_comm_ptr = NULL; - MPIR_Group *group_ptr = NULL, *new_group_ptr = NULL; - int i = 0, j = 0; - - /* Get comm_ptr for MPI_COMM_WORLD and get the group_ptr for it */ - MPIR_Comm_get_ptr( MPI_COMM_WORLD, comm_ptr ); - mpi_errno = MPIR_Comm_group_impl(comm_ptr, &group_ptr); - if (mpi_errno) - { - TRACE_ERR("Error while creating group_ptr from MPI_COMM_WORLD in MPIDI_Comm_create_from_pami_geom\n"); - return PAMI_ERROR; - } - - /* Create the ranks list from the pami_geometry_range_t array */ - for(i = 0; i < slice_count; i++) - { - num_tasks += (task_slices[i].hi - task_slices[i].lo) + 1; - } - ranks = MPL_calloc0(num_tasks, int); - for(i = 0; i < slice_count; i++) - { - int slice_sz = (task_slices[i].hi - task_slices[i].lo) + 1; - int k = 0; - for(k = 0; k < slice_sz; k++) - { - ranks[j] = task_slices[i].lo + k; - j++; - } - } - - /* Now we have all we need to create the new group. Create it */ - mpi_errno = MPIR_Group_incl_impl(group_ptr, num_tasks, ranks, &new_group_ptr); - if (mpi_errno) - { - TRACE_ERR("Error while creating new_group_ptr from group_ptr in MPIDI_Comm_create_from_pami_geom\n"); - return PAMI_ERROR; - } - - /* Now create the communicator using the new_group_ptr */ - mpi_errno = MPIR_Comm_create_intra(comm_ptr, new_group_ptr, &new_comm_ptr); - if (mpi_errno) - { - TRACE_ERR("Error while creating new_comm_ptr from group_ptr in MPIDI_Comm_create_from_pami_geom\n"); - return PAMI_ERROR; - } - - if(new_comm_ptr) - { - /* Get the geometry from the communicator and set the out parameters */ - *geometry = new_comm_ptr->mpid.geometry; - *cookie = new_comm_ptr; - } - else - { - *geometry = PAMI_GEOMETRY_NULL; - *cookie = NULL; - } - - /* Cleanup */ - MPIU_TestFree(&ranks); - - return PAMI_SUCCESS; -} - -pami_result_t MPIDI_Comm_destroy_external(void *comm_ext) -{ - int mpi_errno = 0; - MPIR_Comm* comm_ptr = (MPIR_Comm*)comm_ext; - mpi_errno = MPIR_Comm_free_impl(comm_ptr); - if (mpi_errno) - { - TRACE_ERR("Error while destroying comm_ptr in MPIDI_Comm_destroy_external\n"); - return PAMI_ERROR; - } - return PAMI_SUCCESS; -} - -typedef struct MPIDI_Post_geom_create -{ - pami_work_t state; - size_t context_offset; - pami_client_t client; - pami_configuration_t *configs; - size_t num_configs; - pami_geometry_t newgeom; - pami_geometry_t parent; - unsigned id; - pami_geometry_range_t *ranges; - pami_task_t *tasks; - size_t count; /* count of ranges or tasks */ - pami_event_function fn; - void* cookie; -} MPIDI_Post_geom_create_t; - -typedef struct MPIDI_Post_geom_destroy -{ - pami_work_t state; - pami_client_t client; - pami_geometry_t *geom; - pami_event_function fn; - void *cookie; -} MPIDI_Post_geom_destroy_t; - -static pami_result_t geom_rangelist_create_wrapper(pami_context_t context, void *cookie) -{ - /* I'll need one of these per geometry creation function..... */ - MPIDI_Post_geom_create_t *geom_struct = (MPIDI_Post_geom_create_t *)cookie; - TRACE_ERR("In geom create wrapper\n"); - return PAMI_Geometry_create_taskrange( - geom_struct->client, - geom_struct->context_offset, - geom_struct->configs, - geom_struct->num_configs, - geom_struct->newgeom, - geom_struct->parent, - geom_struct->id, - geom_struct->ranges, - geom_struct->count, - context, - geom_struct->fn, - geom_struct->cookie); - TRACE_ERR("Done in geom create wrapper\n"); -} -static pami_result_t geom_tasklist_create_wrapper(pami_context_t context, void *cookie) -{ - /* I'll need one of these per geometry creation function..... */ - MPIDI_Post_geom_create_t *geom_struct = (MPIDI_Post_geom_create_t *)cookie; - TRACE_ERR("In geom create wrapper\n"); - return PAMI_Geometry_create_tasklist( - geom_struct->client, - geom_struct->context_offset, - geom_struct->configs, - geom_struct->num_configs, - geom_struct->newgeom, - geom_struct->parent, - geom_struct->id, - geom_struct->tasks, - geom_struct->count, - context, - geom_struct->fn, - geom_struct->cookie); - TRACE_ERR("Done in geom create wrapper\n"); -} - -static pami_result_t geom_destroy_wrapper(pami_context_t context, void *cookie) -{ - MPIDI_Post_geom_destroy_t *geom_destroy_struct = (MPIDI_Post_geom_destroy_t *)cookie; - TRACE_ERR("In geom destroy wrapper\n"); - return PAMI_Geometry_destroy( - geom_destroy_struct->client, - geom_destroy_struct->geom, - context, - geom_destroy_struct->fn, - geom_destroy_struct->cookie); - TRACE_ERR("Done in geom destroy wrapper\n"); -} - - - -void MPIDI_Coll_comm_create(MPIR_Comm *comm) -{ - volatile int geom_init = 1; - int i; - MPIDI_Post_geom_create_t geom_post; - - TRACE_ERR("MPIDI_Coll_comm_create enter\n"); - if (!MPIDI_Process.optimized.collectives) - return; - - if(comm->comm_kind != MPIR_COMM_KIND__INTRACOMM) return; - /* Create a geometry */ - - comm->coll_fns = MPL_calloc0(1, MPIR_Collops); - MPID_assert(comm->coll_fns != NULL); - - if(comm->mpid.geometry != MPIDI_Process.world_geometry) - { - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0)) - fprintf(stderr,"world geom: %p parent geom: %p\n", MPIDI_Process.world_geometry, comm->mpid.parent); - TRACE_ERR("Creating subgeom\n"); - /* Change to this at some point */ - - comm->mpid.tasks = NULL; - for(i=1;i<comm->local_size;i++) - { - /* only if sequential tasks should we use a (single) range. - Multi or reordered ranges are inefficient */ - if(MPID_VCR_GET_LPID(comm->vcr, i) != (MPID_VCR_GET_LPID(comm->vcr, i-1) + 1)) { - /* not sequential, use tasklist */ - MPID_VCR_GET_LPIDS(comm, comm->mpid.tasks); - break; - } - } - /* Should we use a range? (no task list set) */ - if(comm->mpid.tasks == NULL) - { - /* one range, {first rank ... last rank} */ - comm->mpid.range.lo = MPID_VCR_GET_LPID(comm->vcr, 0); - comm->mpid.range.hi = MPID_VCR_GET_LPID(comm->vcr, comm->local_size-1); - } - - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0)) - fprintf(stderr,"create geometry tasks %p {%u..%u}\n", comm->mpid.tasks, MPID_VCR_GET_LPID(comm->vcr, 0),MPID_VCR_GET_LPID(comm->vcr, comm->local_size-1)); - - pami_configuration_t config[3]; - size_t numconfigs = 0; -#ifdef HAVE_PAMI_GEOMETRY_NONCONTIG - config[0].name = PAMI_GEOMETRY_NONCONTIG; - if(MPIDI_Process.optimized.memory & MPIR_OPT_LVL_NONCONTIG) - config[0].value.intval = 0; // Disable non-contig, pamid doesn't use pami for non-contig data collectives - else - config[0].value.intval = 1; // Enable non-contig even though pamid doesn't use pami for non-contig data collectives, - // we still possibly want those collectives for other reasons. - ++numconfigs; -#endif - if(MPIDI_Process.optimized.subcomms) - { - config[numconfigs].name = PAMI_GEOMETRY_OPTIMIZE; - config[numconfigs].value.intval = 1; - ++numconfigs; - } -#ifdef HAVE_PAMI_GEOMETRY_MEMORY_OPTIMIZE - if(MPIDI_Process.optimized.memory) - { - config[numconfigs].name = PAMI_GEOMETRY_MEMORY_OPTIMIZE; - config[numconfigs].value.intval = MPIDI_Process.optimized.memory; /* level of optimization */ - ++numconfigs; - } -#endif - - if((MPIDI_Process.optimized.memory & MPIR_OPT_LVL_IRREG) && (comm->local_size & (comm->local_size-1))) - { - /* Don't create irregular geometries. Fallback to MPICH only collectives */ - geom_init = 0; - comm->mpid.geometry = PAMI_GEOMETRY_NULL; - } - else if(comm->mpid.tasks == NULL) - { - geom_post.client = MPIDI_Client; - geom_post.configs = config; - geom_post.context_offset = 0; /* TODO BES investigate */ - geom_post.num_configs = numconfigs; - geom_post.newgeom = &comm->mpid.geometry, - geom_post.parent = PAMI_GEOMETRY_NULL; - geom_post.id = comm->context_id; - geom_post.ranges = &comm->mpid.range; - geom_post.tasks = NULL;; - geom_post.count = (size_t)1; - geom_post.fn = geom_create_cb_done; - geom_post.cookie = (void*)&geom_init; - - TRACE_ERR("%s geom_rangelist_create\n", MPIDI_Process.context_post>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &geom_post.state, - geom_rangelist_create_wrapper, (void *)&geom_post); - } - else - { - geom_post.client = MPIDI_Client; - geom_post.configs = config; - geom_post.context_offset = 0; /* TODO BES investigate */ - geom_post.num_configs = numconfigs; - geom_post.newgeom = &comm->mpid.geometry, - geom_post.parent = PAMI_GEOMETRY_NULL; - geom_post.id = comm->context_id; - geom_post.ranges = NULL; - geom_post.tasks = comm->mpid.tasks; - geom_post.count = (size_t)comm->local_size; - geom_post.fn = geom_create_cb_done; - geom_post.cookie = (void*)&geom_init; - - TRACE_ERR("%s geom_tasklist_create\n", MPIDI_Process.context_post>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &geom_post.state, - geom_tasklist_create_wrapper, (void *)&geom_post); - } - - TRACE_ERR("Waiting for geom create to finish\n"); - MPID_PROGRESS_WAIT_WHILE(geom_init); - - if(comm->mpid.geometry == PAMI_GEOMETRY_NULL) - { - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0)) - fprintf(stderr,"Created unoptimized communicator id=%u, size=%u\n", (unsigned) comm->context_id,comm->local_size); - MPIU_TestFree(&comm->coll_fns); - return; - } - } - /* Initialize the async flow control in case it will be used. */ - comm->mpid.num_requests = MPIDI_Process.optimized.num_requests; - - TRACE_ERR("Querying protocols\n"); - /* Determine what protocols are available for this comm/geom */ - /* These two functions moved to mpid_collselect.c */ - MPIDI_Comm_coll_query(comm); - MPIDI_Comm_coll_envvars(comm); - if(MPIDI_Process.optimized.select_colls) - MPIDI_Comm_coll_select(comm); - TRACE_ERR("mpir barrier\n"); - int mpierrno = FALSE; - /* Switch to comm->coll_fns->fn() */ - MPIDO_Barrier(comm, &mpierrno); - - - TRACE_ERR("MPIDI_Coll_comm_create exit\n"); -} - -void MPIDI_Coll_comm_destroy(MPIR_Comm *comm) -{ - TRACE_ERR("MPIDI_Coll_comm_destroy enter\n"); - int i; - volatile int geom_destroy = 1; - MPIDI_Post_geom_destroy_t geom_destroy_post; - if (!MPIDI_Process.optimized.collectives) - return; - - if(comm->comm_kind != MPIR_COMM_KIND__INTRACOMM) - return; - - /* It's possible (MPII_Setup_intercomm_localcomm) to have an intracomm - without a geometry even when using optimized collectives */ - if(comm->mpid.geometry == PAMI_GEOMETRY_NULL) - return; - - MPIU_TestFree(&comm->coll_fns); - for(i=0;i<PAMI_XFER_COUNT;i++) - { - TRACE_ERR("Freeing algo/meta %d\n", i); - /* When allocating comm->mpid.coll_algorithm, we skip allocations for - AM collectives. Also there is no explicit initialization of - comm->mpid.coll_algorithm to NULLs. This may cause MPIU_TestFree to - cause problems when freeing. We skip AM collectives here as we skip - allocating them in MPIDI_Comm_coll_query */ - if(i == PAMI_XFER_AMBROADCAST || i == PAMI_XFER_AMSCATTER || - i == PAMI_XFER_AMGATHER || i == PAMI_XFER_AMREDUCE) - continue; - - MPIU_TestFree(&comm->mpid.coll_algorithm[i][0]); - MPIU_TestFree(&comm->mpid.coll_algorithm[i][1]); - MPIU_TestFree(&comm->mpid.coll_metadata[i][0]); - MPIU_TestFree(&comm->mpid.coll_metadata[i][1]); - } - - - if(MPIDI_Process.optimized.auto_select_colls != MPID_AUTO_SELECT_COLLS_NONE && MPIDI_Process.optimized.auto_select_colls != MPID_AUTO_SELECT_COLLS_TUNE && comm->local_size > 1) - { - /* Destroy the fast query object. */ - pami_extension_collsel_query_destroy pamix_collsel_query_destroy = - (pami_extension_collsel_query_destroy) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_query_destroy"); - if(pamix_collsel_query_destroy != NULL) - { - pamix_collsel_query_destroy(&(comm->mpid.collsel_fast_query)); - } - } - - TRACE_ERR("Destroying geometry\n"); - - geom_destroy_post.client = MPIDI_Client; - geom_destroy_post.geom = &comm->mpid.geometry; - geom_destroy_post.fn = geom_destroy_cb_done; - geom_destroy_post.cookie = (void *)&geom_destroy; - TRACE_ERR("%s geom_destroy\n", MPIDI_Process.context_post>0?"Posting":"Invoking"); - MPIDI_Context_post(MPIDI_Context[0], &geom_destroy_post.state, - geom_destroy_wrapper, (void *)&geom_destroy_post); - - TRACE_ERR("Waiting for geom destroy to finish\n"); - MPID_PROGRESS_WAIT_WHILE(geom_destroy); - MPID_VCR_FREE_LPIDS(comm->mpid.tasks); -/* TRACE_ERR("Freeing geometry ranges\n"); - MPIU_TestFree(&comm->mpid.tasks_descriptor.ranges); -*/ - TRACE_ERR("MPIDI_Coll_comm_destroy exit\n"); -} diff --git a/src/mpid/pamid/src/comm/mpid_optcolls.c b/src/mpid/pamid/src/comm/mpid_optcolls.c deleted file mode 100644 index d64ed04..0000000 --- a/src/mpid/pamid/src/comm/mpid_optcolls.c +++ /dev/null @@ -1,1047 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/comm/mpid_optcolls.c - * \brief Code for setting up optimized collectives per geometry/communicator - * \brief currently uses strcmp and known protocol performance - */ - - - -/*#define TRACE_ON */ - -#include <mpidimpl.h> - - -static int MPIDI_Check_FCA_envvar(char *string, int *user_range_hi) -{ - char *env = getenv("MP_MPI_PAMI_FOR"); - if(env != NULL) - { - if(strcasecmp(env, "ALL") == 0) - return 1; - int len = strlen(env); - len++; - char *temp = MPL_malloc(sizeof(char) * len); - char *ptrToFree = temp; - strcpy(temp, env); - char *sepptr; - for(sepptr = temp; (sepptr = strsep(&temp, ",")) != NULL ; ) - { - char *subsepptr, *temp_sepptr; - temp_sepptr = sepptr; - subsepptr = strsep(&temp_sepptr, ":"); - if(temp_sepptr != NULL)/* SSS: There is a a colon for this collective */ - { - if(strcasecmp(subsepptr, string) == 0) - { - *user_range_hi = atoi(temp_sepptr); - MPL_free(ptrToFree); - return 1; - } - else - sepptr++; - } - else - { - if(strcasecmp(sepptr, string) == 0) - { - *user_range_hi = -1; - MPL_free(ptrToFree); - return 1; - } - else - sepptr++; - } - } - /* We didn't find it, but the end var was set, so return 0 */ - MPL_free(ptrToFree); - return 0; - } - if(MPIDI_Process.optimized.collectives == MPID_COLL_FCA) - return 1; /* To have gotten this far, opt colls are on. If the env doesn't exist, - we should use the FCA protocol for "string" */ - else - return -1; /* we don't have any FCA things */ -} - -static inline void -MPIDI_Coll_comm_check_FCA(char *coll_name, - char *protocol_name, - int pami_xfer, - int query_type, - int proto_num, - MPIR_Comm *comm_ptr) -{ - int opt_proto = -1; - int i; - int user_range_hi = -1;/* SSS: By default we assume user hasn't defined a range_hi (cutoff_size) */ -#ifdef TRACE_ON - char *envstring = getenv("MP_MPI_PAMI_FOR"); -#endif - TRACE_ERR("Checking for %s in %s\n", coll_name, envstring); - int check_var = MPIDI_Check_FCA_envvar(coll_name, &user_range_hi); - if(check_var == 1) - { - TRACE_ERR("Found %s\n",coll_name); - /* Look for protocol_name in the "always works list */ - for(i = 0; i <comm_ptr->mpid.coll_count[pami_xfer][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[pami_xfer][0][i].name, protocol_name) == 0) - { - opt_proto = i; - break; - } - } - if(opt_proto != -1) /* we found it, so copy it to the optimized var */ - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimized protocol\n", - pami_xfer, opt_proto, - comm_ptr->mpid.coll_metadata[pami_xfer][0][opt_proto].name); - comm_ptr->mpid.opt_protocol[pami_xfer][proto_num] = - comm_ptr->mpid.coll_algorithm[pami_xfer][0][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[pami_xfer][proto_num], - &comm_ptr->mpid.coll_metadata[pami_xfer][0][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[pami_xfer][proto_num] = query_type; - if(user_range_hi != -1) - comm_ptr->mpid.cutoff_size[pami_xfer][proto_num] = user_range_hi; - else - comm_ptr->mpid.cutoff_size[pami_xfer][proto_num] = 0; - comm_ptr->mpid.user_selected_type[pami_xfer] = MPID_COLL_OPTIMIZED; - } - else /* see if it is in the must query list instead */ - { - for(i = 0; i <comm_ptr->mpid.coll_count[pami_xfer][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[pami_xfer][1][i].name, protocol_name) == 0) - { - opt_proto = i; - break; - } - } - if(opt_proto != -1) /* ok, it was in the must query list */ - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimized protocol\n", - pami_xfer, opt_proto, - comm_ptr->mpid.coll_metadata[pami_xfer][1][opt_proto].name); - comm_ptr->mpid.opt_protocol[pami_xfer][proto_num] = - comm_ptr->mpid.coll_algorithm[pami_xfer][1][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[pami_xfer][proto_num], - &comm_ptr->mpid.coll_metadata[pami_xfer][1][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[pami_xfer][proto_num] = query_type; - if(user_range_hi != -1) - comm_ptr->mpid.cutoff_size[pami_xfer][proto_num] = user_range_hi; - else - comm_ptr->mpid.cutoff_size[pami_xfer][proto_num] = 0; - comm_ptr->mpid.user_selected_type[pami_xfer] = MPID_COLL_OPTIMIZED; - } - else /* that protocol doesn't exist */ - { - TRACE_ERR("Couldn't find %s in the list for %s, reverting to MPICH\n",protocol_name,coll_name); - comm_ptr->mpid.user_selected_type[pami_xfer] = MPID_COLL_USE_MPICH; - } - } - } - else if(check_var == 0)/* The env var was set, but wasn't set for coll_name */ - { - TRACE_ERR("Couldn't find any optimal %s protocols or user chose not to set it. Selecting MPICH\n",coll_name); - comm_ptr->mpid.user_selected_type[pami_xfer] = MPID_COLL_USE_MPICH; - } - else - return; -} - - -void MPIDI_Comm_coll_select(MPIR_Comm *comm_ptr) -{ - TRACE_ERR("Entering MPIDI_Comm_coll_select\n"); - int opt_proto = -1; - int mustquery = 0; - int i; - int use_threaded_collectives = 1; - - /* Some highly optimized protocols (limited resource) do not - support MPI_THREAD_MULTIPLE semantics so do not enable them - except on COMM_WORLD. - NOTE: we are not checking metadata because these are known, - hardcoded optimized protocols. - */ - if((MPIR_ThreadInfo.thread_provided == MPI_THREAD_MULTIPLE) && - (comm_ptr != MPIR_Process.comm_world)) use_threaded_collectives = 0; - if(unlikely(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm_ptr->rank == 0)) - fprintf(stderr, "thread_provided=%s, %scomm_world, use_threaded_collectives %u\n", - MPIR_ThreadInfo.thread_provided == MPI_THREAD_MULTIPLE? "MPI_THREAD_MULTIPLE": - MPIR_ThreadInfo.thread_provided == MPI_THREAD_SINGLE?"MPI_THREAD_SINGLE": - MPIR_ThreadInfo.thread_provided == MPI_THREAD_FUNNELED?"MPI_THREAD_FUNNELED": - MPIR_ThreadInfo.thread_provided == MPI_THREAD_SERIALIZED?"MPI_THREAD_SERIALIZED": - "??", - (comm_ptr != MPIR_Process.comm_world)?"!":"", - use_threaded_collectives); - - /* First, setup the (easy, allreduce is complicated) FCA collectives if there - * are any because they are always usable when they are on */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_REDUCE] == MPID_COLL_NOSELECTION) - { - MPIDI_Coll_comm_check_FCA("REDUCE","I1:Reduce:FCA:FCA",PAMI_XFER_REDUCE,MPID_COLL_CHECK_FN_REQUIRED, 0, comm_ptr); - } - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLGATHER] == MPID_COLL_NOSELECTION) - { - MPIDI_Coll_comm_check_FCA("ALLGATHER","I1:Allgather:FCA:FCA",PAMI_XFER_ALLGATHER,MPID_COLL_NOQUERY, 0, comm_ptr); - } - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLGATHERV_INT] == MPID_COLL_NOSELECTION) - { - MPIDI_Coll_comm_check_FCA("ALLGATHERV","I1:AllgathervInt:FCA:FCA",PAMI_XFER_ALLGATHERV_INT,MPID_COLL_NOQUERY, 0, comm_ptr); - } - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] == MPID_COLL_NOSELECTION) - { - MPIDI_Coll_comm_check_FCA("BCAST", "I1:Broadcast:FCA:FCA", PAMI_XFER_BROADCAST, MPID_COLL_NOQUERY, 0, comm_ptr); - MPIDI_Coll_comm_check_FCA("BCAST", "I1:Broadcast:FCA:FCA", PAMI_XFER_BROADCAST, MPID_COLL_NOQUERY, 1, comm_ptr); - } - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BARRIER] == MPID_COLL_NOSELECTION) - { - MPIDI_Coll_comm_check_FCA("BARRIER","I1:Barrier:FCA:FCA",PAMI_XFER_BARRIER,MPID_COLL_NOQUERY, 0, comm_ptr); - } - /* SSS: There isn't really an FCA Gatherv protocol. We do this call to force the use of MPICH for gatherv - * when FCA is enabled so we don't have to use PAMI protocol. */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_GATHERV_INT] == MPID_COLL_NOSELECTION) - { - MPIDI_Coll_comm_check_FCA("GATHERV","I1:GathervInt:FCA:FCA",PAMI_XFER_GATHERV_INT,MPID_COLL_NOQUERY, 0, comm_ptr); - } - - opt_proto = -1; - mustquery = 0; - /* So, several protocols are really easy. Tackle them first. */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLGATHERV_INT] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("No allgatherv[int] env var, so setting optimized allgatherv[int]\n"); - /* Use I0:RectangleDput */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_ALLGATHERV_INT][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLGATHERV_INT][1][i].name, "I0:RectangleDput:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d number %d (%s) to optimized protocol\n", - PAMI_XFER_ALLGATHERV_INT, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLGATHERV_INT][mustquery][opt_proto].name); - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLGATHERV_INT][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_ALLGATHERV_INT][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLGATHERV_INT][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLGATHERV_INT][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_ALLGATHERV_INT][0] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLGATHERV_INT] = MPID_COLL_OPTIMIZED; - } - else /* no optimized allgatherv? */ - { - TRACE_ERR("Couldn't find optimial allgatherv[int] protocol\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLGATHERV_INT] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLGATHERV_INT][0] = 0; - comm_ptr->mpid.allgathervs[0] = 1; /* Use GLUE_ALLREDUCE */ - } - TRACE_ERR("Done setting optimized allgatherv[int]\n"); - } - - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_GATHER] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("Default gather to MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_GATHER] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_GATHER][0] = 0; - } - - opt_proto = -1; - mustquery = 0; - /* Scatter */ - /* If the user has forced a selection, don't bother setting it here */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTER] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("No scatter env var, so setting optimized scatter\n"); - /* The best scatter seems to be I0:Binomial (at least better than I0:Flat) */ - /* Loop over the protocols until we find the one we want */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_SCATTER][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTER][0][i].name, "I0:Binomial:-:MU") == 0) - { - opt_proto = i; - break; - } - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTER][0][i].name, "I0:Binomial:SHMEM:MU") == 0) - { - opt_proto = i; - break; - } - } - if(opt_proto == -1) /* check other list */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_SCATTER][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTER][1][i].name, "I0:Binomial:-:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTER][1][i].name, "I0:Binomial:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - /* Actually, MPICH looks better right now, so use MPICH - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimized protocol\n", - PAMI_XFER_SCATTER, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTER][mustquery][opt_proto].name); - comm_ptr->mpid.opt_protocol[PAMI_XFER_SCATTER][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_SCATTER][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_SCATTER][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTER][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_SCATTER][0] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTER] = MPID_COLL_OPTIMIZED; - } - else - */ - { - TRACE_ERR("Scatter reverting to MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTER] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_SCATTER][0] = 0; - } - TRACE_ERR("Done setting optimized scatter\n"); - } - opt_proto = -1; - mustquery = 0; - /* Scatterv */ - /* If the user has forced a selection, don't bother setting it here */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTERV_INT] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("No scatterv env var, so setting optimized scatterv\n"); - /* The best scatterv seems to be I0:SoftwareTree: */ - /* Loop over the protocols until we find the one we want */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_SCATTERV_INT][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTERV_INT][0][i].name, "I0:SoftwareTree:-:MU") == 0) - { - opt_proto = i; - break; - } - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTERV_INT][0][i].name, "I0:SoftwareTree:SHMEM:MU") == 0) - { - opt_proto = i; - break; - } - } - if(opt_proto == -1) /* check other list */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_SCATTERV_INT][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTERV_INT][1][i].name, "I0:SoftwareTree:-:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTERV_INT][1][i].name, "I0:SoftwareTree:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimized protocol\n", - PAMI_XFER_SCATTERV_INT, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTERV_INT][mustquery][opt_proto].name); - comm_ptr->mpid.opt_protocol[PAMI_XFER_SCATTERV_INT][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_SCATTERV_INT][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_SCATTERV_INT][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_SCATTERV_INT][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_SCATTERV_INT][0] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTERV_INT] = MPID_COLL_OPTIMIZED; - } - else - { - TRACE_ERR("Scatterv reverting to MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTERV_INT] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_SCATTERV_INT][0] = 0; - } - TRACE_ERR("Done setting optimized scatterv\n"); - } - - - opt_proto = -1; - mustquery = 0; - /* Alltoall */ - /* If the user has forced a selection, don't bother setting it here */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALL] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("No alltoall env var, so setting optimized alltoall\n"); - /* The best alltoall is always I0:M2MComposite:MU:MU */ - /* Loop over the protocols until we find the one we want */ - if(use_threaded_collectives) - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_ALLTOALL][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALL][0][i].name, "I0:M2MComposite:MU:MU") == 0) - { - opt_proto = i; - break; - } - } - if(use_threaded_collectives && (opt_proto == -1)) /* check other list */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_ALLTOALL][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALL][1][i].name, "I0:M2MComposite:MU:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimized protocol\n", - PAMI_XFER_ALLTOALL, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALL][mustquery][opt_proto].name); - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLTOALL][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_ALLTOALL][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLTOALL][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALL][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_ALLTOALL][0] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALL] = MPID_COLL_OPTIMIZED; - } - else - { - TRACE_ERR("Couldn't find I0:M2MComposite:MU:MU in the list for alltoall, reverting to MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALL] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLTOALL][0] = 0; - } - TRACE_ERR("Done setting optimized alltoall\n"); - } - - - opt_proto = -1; - mustquery = 0; - /* Alltoallv */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALLV_INT] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("No alltoallv env var, so setting optimized alltoallv\n"); - /* The best alltoall is always I0:M2MComposite:MU:MU */ - /* Loop over the protocols until we find the one we want */ - if(use_threaded_collectives) - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_ALLTOALLV_INT][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALLV_INT][0][i].name, "I0:M2MComposite:MU:MU") == 0) - { - opt_proto = i; - break; - } - } - if(use_threaded_collectives && (opt_proto == -1)) /* check other list */ - for(i = 0; i <comm_ptr->mpid.coll_count[PAMI_XFER_ALLTOALLV_INT][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALLV_INT][1][i].name, "I0:M2MComposite:MU:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimized protocol\n", - PAMI_XFER_ALLTOALLV_INT, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALLV_INT][mustquery][opt_proto].name); - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLTOALLV_INT][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_ALLTOALLV_INT][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLTOALLV_INT][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLTOALLV_INT][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_ALLTOALLV_INT][0] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALLV_INT] = MPID_COLL_OPTIMIZED; - } - else - { - TRACE_ERR("Couldn't find I0:M2MComposite:MU:MU in the list for alltoallv, reverting to MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALLV_INT] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLTOALLV_INT][0] = 0; - } - TRACE_ERR("Done setting optimized alltoallv\n"); - } - - opt_proto = -1; - mustquery = 0; - /* Barrier */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BARRIER] == MPID_COLL_NOSELECTION) - { - TRACE_ERR("No barrier env var, so setting optimized barrier\n"); - /* For 1ppn, I0:MultiSync:-:GI is best, followed by - * I0:RectangleMultiSync:-:MU, followed by - * I0:OptBinomial:P2P:P2P - */ - /* For 16 and 64 ppn, I0:MultiSync2Device:SHMEM:GI (which doesn't exist at 1ppn) - * is best, followed by - * I0:RectangleMultiSync2Device:SHMEM:MU for rectangles, followed by - * I0:OptBinomial:P2P:P2P */ - /* So we basically check for the protocols in reverse-optimal order */ - - - /* In order, >1ppn we use - * I0:MultiSync2Device:SHMEM:GI - * I0:RectangleMultiSync2Device:SHMEM:MU - * I0:OptBinomial:P2P:P2P - * MPICH - * - * In order 1ppn we use - * I0:MultiSync:-:GI - * I0:RectangleMultiSync:-:MU - * I0:OptBinomial:P2P:P2P - * MPICH - */ - if(use_threaded_collectives) - for(i = 0 ; i < comm_ptr->mpid.coll_count[PAMI_XFER_BARRIER][0]; i++) - { - /* These two are mutually exclusive */ - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][i].name, "I0:MultiSync2Device:SHMEM:GI") == 0) - { - opt_proto = i; - } - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][i].name, "I0:MultiSync:-:GI") == 0) - { - opt_proto = i; - } - } - /* Next best rectangular to check */ - if(opt_proto == -1) - { - if(use_threaded_collectives) - for(i = 0 ; i < comm_ptr->mpid.coll_count[PAMI_XFER_BARRIER][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][i].name, "I0:RectangleMultiSync2Device:SHMEM:MU") == 0) - opt_proto = i; - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][i].name, "I0:RectangleMultiSync:-:MU") == 0) - opt_proto = i; - } - } - /* Finally, see if we have opt binomial */ - if(opt_proto == -1) - { - for(i = 0 ; i < comm_ptr->mpid.coll_count[PAMI_XFER_BARRIER][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][i].name, "I0:OptBinomial:P2P:P2P") == 0) - opt_proto = i; - } - } - - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimize protocol\n", - PAMI_XFER_BARRIER, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][opt_proto].name); - - comm_ptr->mpid.opt_protocol[PAMI_XFER_BARRIER][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_BARRIER][0][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BARRIER][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_BARRIER][0][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_BARRIER][0] = MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_BARRIER] = MPID_COLL_OPTIMIZED; - } - else - { - TRACE_ERR("Couldn't find any optimal barrier protocols. Selecting MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_BARRIER] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_BARRIER][0] = 0; - } - - TRACE_ERR("Done setting optimized barrier\n"); - } - - opt_proto = -1; - mustquery = 0; - - /* This becomes messy when we have to message sizes. If we were gutting the - * existing framework, it might be easier, but I think the existing framework - * is useful for future collective selection libraries/mechanisms, so I'd - * rather leave it in place and deal with it here instead.... */ - - /* Broadcast */ - /* 1ppn */ - /* small messages: I0:MulticastDput:-:MU - * if it exists, I0:RectangleDput:MU:MU for >64k */ - - /* 16ppn */ - /* small messages: I0:MultiCastDput:SHMEM:MU - * for 16ppn/1node: I0:MultiCastDput:SHMEM:- perhaps - * I0:RectangleDput:SHMEM:MU for >128k */ - /* nonrect(?): I0:MultiCast2DeviceDput:SHMEM:MU - * no hw: I0:Sync2-nary:Shmem:MUDput */ - /* 64ppn */ - /* all sizes: I0:MultiCastDput:SHMEM:MU */ - /* otherwise, I0:2-nomial:SHMEM:MU */ - - - - /* First, set up small message bcasts */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] == MPID_COLL_NOSELECTION) - { - /* Complicated exceptions: */ - /* I0:RankBased_Binomial:-:ShortMU is good on irregular for <256 bytes */ - /* I0:MultiCast:SHMEM:- is good at 1 node/16ppn, which is a SOW point */ - TRACE_ERR("No bcast env var, so setting optimized bcast\n"); - - if(use_threaded_collectives) - for(i = 0 ; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - /* These two are mutually exclusive */ - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:MultiCastDput:-:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:MultiCastDput:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - /* Next best MU 2 device to check */ - if(use_threaded_collectives) - if(opt_proto == -1) - { - if(use_threaded_collectives) - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:MultiCast2DeviceDput:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - break; - } - } - } - /* Check for rectangle */ - if(use_threaded_collectives) - if(opt_proto == -1) - { - unsigned len = strlen("I0:RectangleDput:"); - if(use_threaded_collectives) - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - if(strcasecmp (comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:RectangleDput:SHMEM:MU") == 0) - { /* Prefer the :SHMEM:MU so break when it's found */ - opt_proto = i; - mustquery = 1; - break; - } - /* Otherwise any RectangleDput is better than nothing. */ - if(strncasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:RectangleDput:",len) == 0) - { - opt_proto = i; - mustquery = 1; - } - } - } - if(opt_proto == -1) - { - /* this protocol is sometimes query, sometimes always works so check both lists but prefer always works */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][0]; i++) - { - /* This is a good choice for small messages only */ - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][0][i].name, "I0:SequenceBased_Binomial:SHMEM:MU") == 0) - { - opt_proto = i; - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0] = 256; - break; - } - } - if(opt_proto == -1) for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - /* This is a good choice for small messages only */ - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:SequenceBased_Binomial:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0] = 256; - break; - } - } - } - /* Next best to check */ - if(opt_proto == -1) - { - /* this protocol is sometimes query, sometimes always works so check both lists but prefer always works */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][0][i].name, "I0:2-nomial:SHMEM:MU") == 0) - opt_proto = i; - } - if(opt_proto == -1) for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:2-nomial:SHMEM:MU") == 0) - opt_proto = i; - mustquery = 1; - } - } - - /* These protocols are good for most message sizes, but there are some - * better choices for larger messages */ - /* Set opt_proto for bcast[0] right now */ - if(opt_proto != -1) - { - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimize protocol 0\n", - PAMI_XFER_BROADCAST, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][mustquery][opt_proto].name); - - comm_ptr->mpid.opt_protocol[PAMI_XFER_BROADCAST][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_BROADCAST][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][0] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] = MPID_COLL_OPTIMIZED; - } - else - { - TRACE_ERR("Couldn't find any optimal bcast protocols. Selecting MPICH\n"); - comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.opt_protocol[PAMI_XFER_BROADCAST][0] = 0; - } - - TRACE_ERR("Done setting optimized bcast 0\n"); - - /* Now, look into large message bcasts */ - opt_proto = -1; - mustquery = 0; - /* If bcast0 is I0:MultiCastDput:-:MU, and I0:RectangleDput:MU:MU is available, use - * it for 64k messages */ - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] != MPID_COLL_USE_MPICH) - { - if(use_threaded_collectives) - if(strcasecmp(comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0].name, "I0:MultiCastDput:-:MU") == 0) - { - /* See if I0:RectangleDput:MU:MU is available */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:RectangleDput:MU:MU") == 0) - { - opt_proto = i; - mustquery = 1; - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0] = 65536; - break; - } - } - } - /* small messages: I0:MultiCastDput:SHMEM:MU*/ - /* I0:RectangleDput:SHMEM:MU for >128k */ - if(use_threaded_collectives) - if(strcasecmp(comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0].name, "I0:MultiCastDput:SHMEM:MU") == 0) - { - /* See if I0:RectangleDput:SHMEM:MU is available */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:RectangleDput:SHMEM:MU") == 0) - { - opt_proto = i; - mustquery = 1; - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0] = 131072; - break; - } - } - } - if(strcasecmp(comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0].name, "I0:SequenceBased_Binomial:SHMEM:MU") == 0) - { - /* This protocol was only good for up to 256, and it was an irregular, so let's set - * 2-nomial for larger message sizes. Cutoff should have already been set to 256 too */ - /* this protocol is sometimes query, sometimes always works so check both lists but prefer always works */ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][0]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][0][i].name, "I0:2-nomial:SHMEM:MU") == 0) - opt_proto = i; - } - if(opt_proto == -1) for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_BROADCAST][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][1][i].name, "I0:2-nomial:SHMEM:MU") == 0) - opt_proto = i; - mustquery = 1; - } - } - - if(opt_proto != -1) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm_ptr->rank == 0) - { - fprintf(stderr,"Selecting %s as optimal broadcast 1 (above %d)\n", - comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][mustquery][opt_proto].name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0]); - } - TRACE_ERR("Memcpy protocol type %d, number %d (%s) to optimize protocol 1 (above %d)\n", - PAMI_XFER_BROADCAST, opt_proto, - comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][mustquery][opt_proto].name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0]); - - comm_ptr->mpid.opt_protocol[PAMI_XFER_BROADCAST][1] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_BROADCAST][mustquery][opt_proto]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][1], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_BROADCAST][mustquery][opt_proto], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][1] = mustquery?MPID_COLL_ALWAYS_QUERY:MPID_COLL_NOQUERY; - /* This should already be set... */ - /* comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] = MPID_COLL_OPTIMIZED; */ - } - else - { - TRACE_ERR("Secondary bcast protocols unavilable; using primary for all sizes\n"); - - TRACE_ERR("Duplicating protocol type %d, number %d (%s) to optimize protocol 1 (above %d)\n", - PAMI_XFER_BROADCAST, 0, - comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0].name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0]); - - comm_ptr->mpid.opt_protocol[PAMI_XFER_BROADCAST][1] = - comm_ptr->mpid.opt_protocol[PAMI_XFER_BROADCAST][0]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][1], - &comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0], - sizeof(pami_metadata_t)); - comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][1] = comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][0]; - } - } - TRACE_ERR("Done with bcast protocol selection\n"); - } - - opt_proto = -1; - mustquery = 0; - /* The most fun... allreduce */ - /* 512-way data: */ - /* For starters, Amith's protocol works on doubles on sum/min/max. Because - * those are targetted types/ops, we will pre-cache it. - * That protocol works on ints, up to 8k/ppn max message size. We'll precache - * it too - */ - - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLREDUCE] == MPID_COLL_NOSELECTION) - { - /* the user hasn't selected a protocol, so we can NULL the protocol/metadatas */ - comm_ptr->mpid.query_cached_allreduce = MPID_COLL_USE_MPICH; - - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0] = 0; - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][0] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][1] = MPID_COLL_USE_MPICH; - /* For BGQ */ - /* 1ppn: I0:MultiCombineDput:-:MU if it is available, but it has a check_fn - * since it is MU-based*/ - /* Next best is I1:ShortAllreduce:P2P:P2P for short messages, then MPICH is best*/ - /* I0:MultiCombineDput:-:MU could be used in the i/dsmm cached protocols, so we'll do that */ - /* First, look in the 'must query' list and see i I0:MultiCombine:Dput:-:MU is there */ - - char *pname="...none..."; - int user_range_hi = -1; - int fca_enabled = 0; - if((fca_enabled = MPIDI_Check_FCA_envvar("ALLREDUCE", &user_range_hi)) == 1) - pname = "I1:Allreduce:FCA:FCA"; - else if(use_threaded_collectives) - pname = "I0:MultiCombineDput:-:MU"; - /*SSS: Any "MU" protocol will not be available on non-BG systems. I just need to check for FCA in the - first if only. No need to do another check since the second if will never succeed for PE systems*/ - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_ALLREDUCE][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLREDUCE][1][i].name, pname) == 0) - { - /* So, this should be fine for the i/dsmm protocols. everything else needs to call the check function */ - /* This also works for all message sizes, so no need to deal with it specially for query */ - comm_ptr->mpid.cached_allreduce = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_ALLREDUCE][1][i]; - memcpy(&comm_ptr->mpid.cached_allreduce_md, - &comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLREDUCE][1][i], - sizeof(pami_metadata_t)); - comm_ptr->mpid.query_cached_allreduce = MPID_COLL_QUERY; - - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLREDUCE] = MPID_COLL_OPTIMIZED; - if(fca_enabled && user_range_hi != -1) - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0] = user_range_hi; - opt_proto = i; - - } - if(use_threaded_collectives) - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLREDUCE][1][i].name, "I0:MultiCombineDput:SHMEM:MU") == 0) - { - /* This works well for doubles sum/min/max but has trouble with int > 8k/ppn */ - comm_ptr->mpid.cached_allreduce = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_ALLREDUCE][1][i]; - memcpy(&comm_ptr->mpid.cached_allreduce_md, - &comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLREDUCE][1][i], - sizeof(pami_metadata_t)); - comm_ptr->mpid.query_cached_allreduce = MPID_COLL_CHECK_FN_REQUIRED; - - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLREDUCE] = MPID_COLL_OPTIMIZED; - opt_proto = i; - } - } - /* At this point, if opt_proto != -1, we have must-query protocols in the i/dsmm caches */ - /* We should pick a backup, non-must query */ - /* I0:ShortAllreduce:P2P:P2P < 128, then mpich*/ - - /*SSS: ShortAllreduce is available on both BG and PE. I have to pick just one to check for in this case. - However, I need to add FCA for both opt_protocol[0]and[1] to cover all data sizes*/ - if(fca_enabled == 1) - pname = "I1:Allreduce:FCA:FCA"; - else - pname = "I1:ShortAllreduce:P2P:P2P"; - - for(i = 0; i < comm_ptr->mpid.coll_count[PAMI_XFER_ALLREDUCE][1]; i++) - { - if(strcasecmp(comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLREDUCE][1][i].name, pname) == 0) - { - comm_ptr->mpid.opt_protocol[PAMI_XFER_ALLREDUCE][0] = - comm_ptr->mpid.coll_algorithm[PAMI_XFER_ALLREDUCE][1][i]; - memcpy(&comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLREDUCE][0], - &comm_ptr->mpid.coll_metadata[PAMI_XFER_ALLREDUCE][1][i], - sizeof(pami_metadata_t)); - if(fca_enabled == 1) - { - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][0] = MPID_COLL_CHECK_FN_REQUIRED; - if(user_range_hi != -1) - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0] = user_range_hi; - /*SSS: Otherwise another protocol may get selected in mpido_allreduce if we don't set this flag here*/ - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][1] = MPID_COLL_CHECK_FN_REQUIRED; - } - else - { - /* Short is good for up to 512 bytes... but it's a query protocol */ - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][0] = MPID_COLL_QUERY; - /* MPICH above that ... when short query fails */ - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][1] = MPID_COLL_USE_MPICH; - } - comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLREDUCE] = MPID_COLL_OPTIMIZED; - - opt_proto = i; - } - } - if(opt_proto == -1) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm_ptr->rank == 0) - fprintf(stderr,"Optimized allreduce falls back to MPICH\n"); - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][0] = MPID_COLL_USE_MPICH; - comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][1] = MPID_COLL_USE_MPICH; - } - TRACE_ERR("Done setting optimized allreduce protocols\n"); - } - - - if(MPIDI_Process.optimized.select_colls != 2) - { - for(i = 0; i < PAMI_XFER_COUNT; i++) - { - if(i == PAMI_XFER_AMBROADCAST || i == PAMI_XFER_AMSCATTER || - i == PAMI_XFER_AMGATHER || i == PAMI_XFER_AMREDUCE) - continue; - if(comm_ptr->mpid.user_selected_type[i] != MPID_COLL_OPTIMIZED) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm_ptr->rank == 0) - fprintf(stderr, "Collective wasn't selected for type %d,using MPICH (comm %p)\n", i, comm_ptr); - comm_ptr->mpid.user_selected_type[i] = MPID_COLL_USE_MPICH; - } - } - } - - - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm_ptr->rank == 0) - { - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BARRIER] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for opt barrier comm %p\n", comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BARRIER][0].name, comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLGATHERV_INT] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for opt allgatherv comm %p\n", comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLGATHERV_INT][0].name, comm_ptr); - if(comm_ptr->mpid.must_query[PAMI_XFER_ALLGATHERV_INT][0] == MPID_COLL_USE_MPICH) - fprintf(stderr,"Selecting MPICH for allgatherv below %d size comm %p\n", comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLGATHERV_INT][0], comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_GATHER] == MPID_COLL_USE_MPICH) - fprintf(stderr,"Selecting MPICH for gather comm %p\n", comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTER] == MPID_COLL_USE_MPICH) - fprintf(stderr,"Selecting MPICH for scatter comm %p\n", comm_ptr); - else if(comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTER] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for scatter comm %p\n", - comm_ptr->mpid.opt_protocol_md[PAMI_XFER_SCATTER][0].name, - comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTERV_INT] == MPID_COLL_USE_MPICH) - fprintf(stderr,"Selecting MPICH for scatterv comm %p\n", comm_ptr); - else if(comm_ptr->mpid.user_selected_type[PAMI_XFER_SCATTERV_INT] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for scatterv comm %p\n", - comm_ptr->mpid.opt_protocol_md[PAMI_XFER_SCATTERV_INT][0].name, - comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_BROADCAST] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for opt bcast up to size %d comm %p\n", comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][0].name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0], comm_ptr); - if((comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][1] == MPID_COLL_NOQUERY) || - (comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][1] == MPID_COLL_ALWAYS_QUERY)) - fprintf(stderr,"Selecting %s (mustquery=%d) for opt bcast above size %d comm %p\n", - comm_ptr->mpid.opt_protocol_md[PAMI_XFER_BROADCAST][1].name, - comm_ptr->mpid.must_query[PAMI_XFER_BROADCAST][1], - comm_ptr->mpid.cutoff_size[PAMI_XFER_BROADCAST][0], comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALLV_INT] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for opt alltoallv comm %p\n", comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLTOALLV_INT][0].name, comm_ptr); - if(comm_ptr->mpid.user_selected_type[PAMI_XFER_ALLTOALL] == MPID_COLL_OPTIMIZED) - fprintf(stderr,"Selecting %s for opt alltoall comm %p\n", comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLTOALL][0].name, comm_ptr); - if(comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][0] == MPID_COLL_USE_MPICH) - fprintf(stderr,"Selecting MPICH for allreduce below %d size comm %p\n", comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0], comm_ptr); - else - { - if(comm_ptr->mpid.query_cached_allreduce != MPID_COLL_USE_MPICH) - { - fprintf(stderr,"Selecting %s for double sum/min/max ops allreduce, query: %d comm %p\n", - comm_ptr->mpid.cached_allreduce_md.name, comm_ptr->mpid.query_cached_allreduce, comm_ptr); - } - fprintf(stderr,"Selecting %s for other operations allreduce up to %d comm %p\n", - comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLREDUCE][0].name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0], comm_ptr); - } - if(comm_ptr->mpid.must_query[PAMI_XFER_ALLREDUCE][1] == MPID_COLL_USE_MPICH) - fprintf(stderr,"Selecting MPICH for allreduce above %d size comm %p\n", comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0], comm_ptr); - else - { - if(comm_ptr->mpid.query_cached_allreduce != MPID_COLL_USE_MPICH) - { - fprintf(stderr,"Selecting %s for double sum/min/max ops allreduce, above %d query: %d comm %p\n", - comm_ptr->mpid.cached_allreduce_md.name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0], - comm_ptr->mpid.query_cached_allreduce, comm_ptr); - } - else - { - fprintf(stderr,"Selecting MPICH for double sum/min/max ops allreduce, above %d size comm %p\n", - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0], comm_ptr); - } - fprintf(stderr,"Selecting %s for other operations allreduce over %d comm %p\n", - comm_ptr->mpid.opt_protocol_md[PAMI_XFER_ALLREDUCE][1].name, - comm_ptr->mpid.cutoff_size[PAMI_XFER_ALLREDUCE][0], comm_ptr); - } - } - - TRACE_ERR("Leaving MPIDI_Comm_coll_select\n"); - -} - diff --git a/src/mpid/pamid/src/comm/mpid_selectcolls.c b/src/mpid/pamid/src/comm/mpid_selectcolls.c deleted file mode 100644 index 8e0fa36..0000000 --- a/src/mpid/pamid/src/comm/mpid_selectcolls.c +++ /dev/null @@ -1,746 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/comm/mpid_collselect.c - * \brief Code for setting up collectives per geometry/communicator - */ - -/*#define TRACE_ON */ - -#include <mpidimpl.h> - -pami_metadata_t ext_metadata; -advisor_algorithm_t ext_algorithms[1]; -external_algorithm_t ext_algorithm; - -#define MPIDI_UPDATE_COLLSEL_EXT_ALGO(cb,nm,xfer_type) { \ - ext_algorithm.callback = cb; \ - ext_algorithm.cookie = comm; \ - ext_metadata.name = nm; \ - ext_algorithms[0].algorithm.external = ext_algorithm; \ - ext_algorithms[0].metadata = &ext_metadata; \ - ext_algorithms[0].algorithm_type = COLLSEL_EXTERNAL_ALGO; \ - pamix_collsel_register_algorithms(MPIDI_Collsel_advisor_table, \ - comm->mpid.geometry, \ - xfer_type, \ - &ext_algorithms[0], \ - 1); \ -} - - -pami_result_t MPIDI_Register_algorithms_ext(void *cookie, - pami_xfer_type_t collective, - advisor_algorithm_t **algorithms, - size_t *num_algorithms) -{ - external_algorithm_fn callback; - char *algoname; - - switch(collective) - { - case PAMI_XFER_BROADCAST: callback = MPIDO_CSWrapper_bcast; algoname = "EXT:Bcast:P2P:P2P"; break; - case PAMI_XFER_ALLREDUCE: callback = MPIDO_CSWrapper_allreduce; algoname = "EXT:Allreduce:P2P:P2P"; break; - case PAMI_XFER_REDUCE: callback = MPIDO_CSWrapper_reduce; algoname = "EXT:Reduce:P2P:P2P"; break; - case PAMI_XFER_ALLGATHER: callback = MPIDO_CSWrapper_allgather; algoname = "EXT:Allgather:P2P:P2P"; break; - case PAMI_XFER_ALLGATHERV_INT: callback = MPIDO_CSWrapper_allgatherv; algoname = "EXT:Allgatherv:P2P:P2P"; break; - case PAMI_XFER_SCATTER: callback = MPIDO_CSWrapper_scatter; algoname = "EXT:Scatter:P2P:P2P"; break; - case PAMI_XFER_SCATTERV_INT: callback = MPIDO_CSWrapper_scatterv; algoname = "EXT:Scatterv:P2P:P2P"; break; - case PAMI_XFER_GATHER: callback = MPIDO_CSWrapper_gather; algoname = "EXT:Gather:P2P:P2P"; break; - case PAMI_XFER_GATHERV_INT: callback = MPIDO_CSWrapper_gatherv; algoname = "EXT:Gatherv:P2P:P2P"; break; - case PAMI_XFER_BARRIER: callback = MPIDO_CSWrapper_barrier; algoname = "EXT:Barrier:P2P:P2P"; break; - case PAMI_XFER_ALLTOALL: callback = MPIDO_CSWrapper_alltoall; algoname = "EXT:Alltoall:P2P:P2P"; break; - case PAMI_XFER_ALLTOALLV_INT: callback = MPIDO_CSWrapper_alltoallv; algoname = "EXT:Alltoallv:P2P:P2P"; break; - case PAMI_XFER_SCAN: callback = MPIDO_CSWrapper_scan; algoname = "EXT:Scan:P2P:P2P"; break; - case PAMI_XFER_ALLGATHERV: - case PAMI_XFER_SCATTERV: - case PAMI_XFER_GATHERV: - case PAMI_XFER_ALLTOALLV: - case PAMI_XFER_REDUCE_SCATTER: - *num_algorithms = 0; - return PAMI_SUCCESS; - default: return -1; - } - *num_algorithms = 1; - ext_algorithm.callback = callback; - ext_algorithm.cookie = cookie; - ext_metadata.name = algoname; - ext_algorithms[0].algorithm.external = ext_algorithm; - ext_algorithms[0].metadata = &ext_metadata; - ext_algorithms[0].algorithm_type = COLLSEL_EXTERNAL_ALGO; - *algorithms = &ext_algorithms[0]; - return PAMI_SUCCESS; -} - -static char* MPIDI_Coll_type_name(int i) -{ - switch(i) - { - case 0: return("Broadcast"); - case 1: return("Allreduce"); - case 2: return("Reduce"); - case 3: return("Allgather"); - case 4: return("Allgatherv"); - case 5: return("Allgatherv_int"); - case 6: return("Scatter"); - case 7: return("Scatterv"); - case 8: return("Scatterv_int"); - case 9: return("Gather"); - case 10: return("Gatherv"); - case 11: return("Gatherv_int"); - case 12: return("Barrier"); - case 13: return("Alltoall"); - case 14: return("Alltoallv"); - case 15: return("Alltoallv_int"); - case 16: return("Scan"); - case 17: return("Reduce_scatter"); - default: return("AM Collective"); - } -} -static void MPIDI_Update_coll(pami_algorithm_t coll, - int type, - int index, - MPIR_Comm *comm); - -static void MPIDI_Update_coll(pami_algorithm_t coll, - int type, /* must query vs always works */ - int index, - MPIR_Comm *comm) -{ - - comm->mpid.user_selected_type[coll] = type; - TRACE_ERR("Update_coll for protocol %s, type: %d index: %d\n", - comm->mpid.coll_metadata[coll][type][index].name, type, index); - - /* Are we in the 'must query' list? If so determine how "bad" it is */ - if(type == MPID_COLL_QUERY) - { - /* First, is a check always required? */ - if(comm->mpid.coll_metadata[coll][type][index].check_correct.values.checkrequired) - { - TRACE_ERR("Protocol %s check_fn required always\n", comm->mpid.coll_metadata[coll][type][index].name); - /* We must have a check_fn */ - MPID_assert_always(comm->mpid.coll_metadata[coll][type][index].check_fn !=NULL); - comm->mpid.user_selected_type[coll] = MPID_COLL_CHECK_FN_REQUIRED; - } - else if(comm->mpid.coll_metadata[coll][type][index].check_fn != NULL) - { - /* For now, if there's a check_fn we will always call it and not cache. - We *could* be smarter about this eventually. */ - TRACE_ERR("Protocol %s setting to always query/call check_fn\n", comm->mpid.coll_metadata[coll][type][index].name); - comm->mpid.user_selected_type[coll] = MPID_COLL_CHECK_FN_REQUIRED; - } - else /* No check fn but we still need to check metadata bits (query protocol) */ - { - TRACE_ERR("Protocol %s setting to always query/no check_fn\n", comm->mpid.coll_metadata[coll][type][index].name); - comm->mpid.user_selected_type[coll] = MPID_COLL_ALWAYS_QUERY; - } - - } - - comm->mpid.user_selected[coll] = - comm->mpid.coll_algorithm[coll][type][index]; - - memcpy(&comm->mpid.user_metadata[coll], - &comm->mpid.coll_metadata[coll][type][index], - sizeof(pami_metadata_t)); -} - -static void MPIDI_Check_preallreduce(char *env, MPIR_Comm *comm, char *name, int constant) -{ - /* If a given protocol only requires a check for nonlocal conditions and preallreduce - * is turned off, we can consider it a always works protocol instead - */ - char *envopts = getenv(env); - if(envopts != NULL) - { - if(strncasecmp(env, "N", 1) == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Bypassing breallreduce for %s\n", name); - comm->mpid.preallreduces[constant] = 0; - } - } -} -static int MPIDI_Check_protocols(char *names[], MPIR_Comm *comm, char *name, int constant) -{ - int i = 0; - char *envopts; - for(;; ++i) - { - if(names[i] == NULL) - return 0; - envopts = getenv(names[i]); - if(envopts != NULL) - break; - } - /* Now, change it if we have a match */ - if(envopts != NULL) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Checking %s against known %s protocols\n", envopts, name); - /* Check if MPICH was specifically requested */ - if(strcasecmp(envopts, "MPICH") == 0) - { - TRACE_ERR("Selecting MPICH for %s\n", name); - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting MPICH for %d (%s)\n", constant, name); - comm->mpid.user_selected_type[constant] = MPID_COLL_USE_MPICH; - comm->mpid.user_selected[constant] = 0; - return 0; - } - - for(i=0; i < comm->mpid.coll_count[constant][0];i++) - { - if(strncasecmp(envopts, comm->mpid.coll_metadata[constant][0][i].name,strlen(envopts)) == 0) - { - MPIDI_Update_coll(constant, MPID_COLL_NOQUERY, i, comm); - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"setting %s as default %s for comm %p\n", comm->mpid.coll_metadata[constant][0][i].name, name, comm); - return 0; - } - } - for(i=0; i < comm->mpid.coll_count[constant][1];i++) - { - if(strncasecmp(envopts, comm->mpid.coll_metadata[constant][1][i].name,strlen(envopts)) == 0) - { - TRACE_ERR("Calling updatecoll...\n"); - MPIDI_Update_coll(constant, MPID_COLL_QUERY, i, comm); - TRACE_ERR("Done calling update coll\n"); - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"setting (query required protocol) %s as default %s for comm %p\n", comm->mpid.coll_metadata[constant][1][i].name, name, comm); - return 0; - } - } - /* An envvar was specified, so we should probably use MPICH of we can't find - * the specified protocol */ - TRACE_ERR("Specified protocol %s was unavailable; using MPICH for %s\n", envopts, name); - comm->mpid.user_selected_type[constant] = MPID_COLL_USE_MPICH; - comm->mpid.user_selected[constant] = 0; - return 0; - } - /* Looks like we didn't get anything, set NOSELECTION so automated selection can pick something */ - comm->mpid.user_selected_type[constant] = MPID_COLL_NOSELECTION; - comm->mpid.user_selected[constant] = 0; - return 0; -} - -void MPIDI_Comm_coll_envvars(MPIR_Comm *comm) -{ - char *envopts; - int i; - MPID_assert_always(comm!=NULL); - TRACE_ERR("MPIDI_Comm_coll_envvars enter\n"); - - /* Set up always-works defaults */ - for(i = 0; i < PAMI_XFER_COUNT; i++) - { - if(i == PAMI_XFER_AMBROADCAST || i == PAMI_XFER_AMSCATTER || - i == PAMI_XFER_AMGATHER || i == PAMI_XFER_AMREDUCE) - continue; - - /* Initialize to noselection instead of noquery for PE/FCA stuff. Is this the right answer? */ - comm->mpid.user_selected_type[i] = MPID_COLL_NOSELECTION; - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Setting up collective %d on comm %p\n", i, comm); - if(((comm->mpid.coll_count[i][0] == 0) && (comm->mpid.coll_count[i][1] == 0)) || MPIDI_Process.optimized.collectives == MPID_COLL_CUDA) - { - comm->mpid.user_selected_type[i] = MPID_COLL_USE_MPICH; - comm->mpid.user_selected[i] = 0; - } - else if(comm->mpid.coll_count[i][0] != 0) - { - comm->mpid.user_selected[i] = comm->mpid.coll_algorithm[i][0][0]; - memcpy(&comm->mpid.user_metadata[i], &comm->mpid.coll_metadata[i][0][0], - sizeof(pami_metadata_t)); - } - else - { - MPIDI_Update_coll(i, MPID_COLL_QUERY, 0, comm); - /* even though it's a query protocol, say NOSELECTION - so the optcoll selection will override (maybe) */ - comm->mpid.user_selected_type[i] = MPID_COLL_NOSELECTION; - } - } - - - TRACE_ERR("Checking env vars\n"); - - MPIDI_Check_preallreduce("PAMID_COLLECTIVE_ALLGATHER_PREALLREDUCE", comm, "allgather", - MPID_ALLGATHER_PREALLREDUCE); - - MPIDI_Check_preallreduce("PAMID_COLLECTIVE_ALLGATHERV_PREALLREDUCE", comm, "allgatherv", - MPID_ALLGATHERV_PREALLREDUCE); - - MPIDI_Check_preallreduce("PAMID_COLLECTIVE_ALLREDUCE_PREALLREDUCE", comm, "allreduce", - MPID_ALLREDUCE_PREALLREDUCE); - - MPIDI_Check_preallreduce("PAMID_COLLECTIVE_BCAST_PREALLREDUCE", comm, "broadcast", - MPID_BCAST_PREALLREDUCE); - - MPIDI_Check_preallreduce("PAMID_COLLECTIVE_SCATTERV_PREALLREDUCE", comm, "scatterv", - MPID_SCATTERV_PREALLREDUCE); - - { - TRACE_ERR("Checking bcast\n"); - char* names[] = {"PAMID_COLLECTIVE_BCAST", "MP_S_MPI_BCAST", NULL}; - MPIDI_Check_protocols(names, comm, "broadcast", PAMI_XFER_BROADCAST); - } - { - TRACE_ERR("Checking allreduce\n"); - char* names[] = {"PAMID_COLLECTIVE_ALLREDUCE", "MP_S_MPI_ALLREDUCE", NULL}; - MPIDI_Check_protocols(names, comm, "allreduce", PAMI_XFER_ALLREDUCE); - } - { - TRACE_ERR("Checking barrier\n"); - char* names[] = {"PAMID_COLLECTIVE_BARRIER", "MP_S_MPI_BARRIER", NULL}; - MPIDI_Check_protocols(names, comm, "barrier", PAMI_XFER_BARRIER); - } - { - TRACE_ERR("Checking alltaoll\n"); - char* names[] = {"PAMID_COLLECTIVE_ALLTOALL", "MP_S_MPI_ALLTOALL", NULL}; - MPIDI_Check_protocols(names, comm, "alltoall", PAMI_XFER_ALLTOALL); - } - comm->mpid.optreduce = 0; - envopts = getenv("PAMID_COLLECTIVE_REDUCE"); - if(envopts != NULL) - { - if(strcasecmp(envopts, "GLUE_ALLREDUCE") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue allreduce for reduce\n"); - comm->mpid.optreduce = 1; - } - } - /* In addition to glue protocols, check for other PAMI protocols and check for PE now */ - { - TRACE_ERR("Checking reduce\n"); - char* names[] = {"PAMID_COLLECTIVE_REDUCE", "MP_S_MPI_REDUCE", NULL}; - MPIDI_Check_protocols(names, comm, "reduce", PAMI_XFER_REDUCE); - } - { - TRACE_ERR("Checking alltoallv\n"); - char* names[] = {"PAMID_COLLECTIVE_ALLTOALLV", "MP_S_MPI_ALLTOALLV", NULL}; - MPIDI_Check_protocols(names, comm, "alltoallv", PAMI_XFER_ALLTOALLV_INT); - } - { - TRACE_ERR("Checking gatherv\n"); - char* names[] = {"PAMID_COLLECTIVE_GATHERV", "MP_S_MPI_GATHERV", NULL}; - MPIDI_Check_protocols(names, comm, "gatherv", PAMI_XFER_GATHERV_INT); - } - { - TRACE_ERR("Checking scan\n"); - char* names[] = {"PAMID_COLLECTIVE_SCAN", "MP_S_MPI_SCAN", NULL}; - MPIDI_Check_protocols(names, comm, "scan", PAMI_XFER_SCAN); - } - - comm->mpid.scattervs[0] = comm->mpid.scattervs[1] = 0; - - TRACE_ERR("Checking scatterv\n"); - envopts = getenv("PAMID_COLLECTIVE_SCATTERV"); - if(envopts != NULL) - { - if(strcasecmp(envopts, "GLUE_BCAST") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue bcast for scatterv\n"); - comm->mpid.scattervs[0] = 1; - } - else if(strcasecmp(envopts, "GLUE_ALLTOALLV") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue alltoallv for scatterv\n"); - comm->mpid.scattervs[1] = 1; - } - } - { /* In addition to glue protocols, check for other PAMI protocols and check for PE now */ - char* names[] = {"PAMID_COLLECTIVE_SCATTERV", "MP_S_MPI_SCATTERV", NULL}; - MPIDI_Check_protocols(names, comm, "scatterv", PAMI_XFER_SCATTERV_INT); - } - - TRACE_ERR("Checking scatter\n"); - comm->mpid.optscatter = 0; - envopts = getenv("PAMID_COLLECTIVE_SCATTER"); - if(envopts != NULL) - { - if(strcasecmp(envopts, "GLUE_BCAST") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_bcast for scatter\n"); - comm->mpid.optscatter = 1; - } - } - { /* In addition to glue protocols, check for other PAMI protocols and check for PE now */ - char* names[] = {"PAMID_COLLECTIVE_SCATTER", "MP_S_MPI_SCATTER", NULL}; - MPIDI_Check_protocols(names, comm, "scatter", PAMI_XFER_SCATTER); - } - - TRACE_ERR("Checking allgather\n"); - comm->mpid.allgathers[0] = comm->mpid.allgathers[1] = comm->mpid.allgathers[2] = 0; - envopts = getenv("PAMID_COLLECTIVE_ALLGATHER"); - if(envopts != NULL) - { - if(strcasecmp(envopts, "GLUE_ALLREDUCE") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_allreduce for allgather\n"); - comm->mpid.allgathers[0] = 1; - } - - else if(strcasecmp(envopts, "GLUE_BCAST") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_bcast for allgather\n"); - comm->mpid.allgathers[1] = 1; - } - - else if(strcasecmp(envopts, "GLUE_ALLTOALL") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_alltoall for allgather\n"); - comm->mpid.allgathers[2] = 1; - } - } - { /* In addition to glue protocols, check for other PAMI protocols and check for PE now */ - char* names[] = {"PAMID_COLLECTIVE_ALLGATHER", "MP_S_MPI_ALLGATHER", NULL}; - MPIDI_Check_protocols(names, comm, "allgather", PAMI_XFER_ALLGATHER); - } - - TRACE_ERR("Checking allgatherv\n"); - comm->mpid.allgathervs[0] = comm->mpid.allgathervs[1] = comm->mpid.allgathervs[2] = 0; - envopts = getenv("PAMID_COLLECTIVE_ALLGATHERV"); - if(envopts != NULL) - { - if(strcasecmp(envopts, "GLUE_ALLREDUCE") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_allreduce for allgatherv\n"); - comm->mpid.allgathervs[0] = 1; - } - - else if(strcasecmp(envopts, "GLUE_BCAST") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_bcast for allgatherv\n"); - comm->mpid.allgathervs[1] = 1; - } - - else if(strcasecmp(envopts, "GLUE_ALLTOALL") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue_alltoall for allgatherv\n"); - comm->mpid.allgathervs[2] = 1; - } - } - { /* In addition to glue protocols, check for other PAMI protocols and check for PE now */ - char* names[] = {"PAMID_COLLECTIVE_ALLGATHERV", "MP_S_MPI_ALLGATHERV", NULL}; - MPIDI_Check_protocols(names, comm, "allgatherv", PAMI_XFER_ALLGATHERV_INT); - } - - TRACE_ERR("CHecking gather\n"); - comm->mpid.optgather = 0; - envopts = getenv("PAMID_COLLECTIVE_GATHER"); - if(envopts != NULL) - { - if(strcasecmp(envopts, "GLUE_REDUCE") == 0) - { - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - fprintf(stderr,"Selecting glue reduce for gather\n"); - comm->mpid.optgather = 1; - } - } - { /* In addition to glue protocols, check for other PAMI protocols and check for PE now */ - char* names[] = {"PAMID_COLLECTIVE_GATHER", "MP_S_MPI_GATHER", NULL}; - MPIDI_Check_protocols(names, comm, "gather", PAMI_XFER_GATHER); - } - - /* If automatic collective selection is enabled and user didn't specifically overwrite - it, then use auto coll sel.. Otherwise, go through the manual coll sel code path. */ - comm->mpid.collsel_fast_query = NULL; /* Init to NULL.. Should only have a value if we create query */ - if(MPIDI_Process.optimized.auto_select_colls != MPID_AUTO_SELECT_COLLS_NONE && MPIDI_Process.optimized.auto_select_colls != MPID_AUTO_SELECT_COLLS_TUNE && comm->local_size > 1) - { - /* Create a fast query object, cache it on the comm/geometry and use it in each collective */ - pami_extension_collsel_query_create pamix_collsel_query_create = - (pami_extension_collsel_query_create) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_query_create"); - if(pamix_collsel_query_create != NULL) - { - pamix_collsel_query_create(MPIDI_Collsel_advisor_table, comm->mpid.geometry, &(comm->mpid.collsel_fast_query)); - } - - MPIDI_Pamix_collsel_advise = /* Get the function pointer and cache it */ - (pami_extension_collsel_advise) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_advise"); - - pami_extension_collsel_register_algorithms pamix_collsel_register_algorithms = - (pami_extension_collsel_register_algorithms) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_register_algorithms"); - if(pamix_collsel_register_algorithms != NULL) - { - - /* ************ Barrier ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_BARRIER) && - comm->mpid.user_selected_type[PAMI_XFER_BARRIER] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Barrier = MPIDO_Barrier_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_barrier,"EXT:Barrier:P2P:P2P",PAMI_XFER_BARRIER); - } - /* ************ Bcast ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_BCAST) && - comm->mpid.user_selected_type[PAMI_XFER_BROADCAST] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Bcast = MPIDO_Bcast_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_bcast,"EXT:Bcast:P2P:P2P",PAMI_XFER_BROADCAST); - } - /* ************ Allreduce ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_ALLREDUCE) && - comm->mpid.user_selected_type[PAMI_XFER_ALLREDUCE] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Allreduce = MPIDO_Allreduce_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_allreduce,"EXT:Allreduce:P2P:P2P",PAMI_XFER_ALLREDUCE); - } - /* ************ Allgather ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_ALLGATHER) && - comm->mpid.user_selected_type[PAMI_XFER_ALLGATHER] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Allgather = MPIDO_Allgather_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_allgather,"EXT:Allgather:P2P:P2P",PAMI_XFER_ALLGATHER); - } - /* ************ Allgatherv ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_ALLGATHERV) && - comm->mpid.user_selected_type[PAMI_XFER_ALLGATHERV_INT] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Allgatherv = MPIDO_Allgatherv_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_allgatherv,"EXT:Allgatherv:P2P:P2P",PAMI_XFER_ALLGATHERV_INT); - } - /* ************ Scatterv ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_SCATTERV) && - comm->mpid.user_selected_type[PAMI_XFER_SCATTERV_INT] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Scatterv = MPIDO_Scatterv_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_scatterv,"EXT:Scatterv:P2P:P2P",PAMI_XFER_SCATTERV_INT); - } - /* ************ Scatter ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_SCATTER) && - comm->mpid.user_selected_type[PAMI_XFER_SCATTER] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Scatter = MPIDO_Scatter_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_scatter,"EXT:Scatter:P2P:P2P",PAMI_XFER_SCATTER); - } - /* ************ Gather ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_GATHER) && - comm->mpid.user_selected_type[PAMI_XFER_GATHER] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Gather = MPIDO_Gather_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_gather,"EXT:Gather:P2P:P2P",PAMI_XFER_GATHER); - } - /* ************ Alltoallv ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_ALLTOALLV) && - comm->mpid.user_selected_type[PAMI_XFER_ALLTOALLV_INT] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Alltoallv = MPIDO_Alltoallv_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_alltoallv,"EXT:Alltoallv:P2P:P2P",PAMI_XFER_ALLTOALLV_INT); - } - /* ************ Alltoall ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_ALLTOALL) && - comm->mpid.user_selected_type[PAMI_XFER_ALLTOALL] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Alltoall = MPIDO_Alltoall_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_alltoall,"EXT:Alltoall:P2P:P2P",PAMI_XFER_ALLTOALL); - } - /* ************ Gatherv ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_GATHERV) && - comm->mpid.user_selected_type[PAMI_XFER_GATHERV_INT] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Gatherv = MPIDO_Gatherv_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_gatherv,"EXT:Gatherv:P2P:P2P",PAMI_XFER_GATHERV_INT); - } - /* ************ Reduce ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_REDUCE) && - comm->mpid.user_selected_type[PAMI_XFER_REDUCE] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Reduce = MPIDO_Reduce_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_reduce,"EXT:Reduce:P2P:P2P",PAMI_XFER_REDUCE); - } - /* ************ Scan ************ */ - if((MPIDI_Process.optimized.auto_select_colls & MPID_AUTO_SELECT_COLLS_SCAN) && - comm->mpid.user_selected_type[PAMI_XFER_SCAN] == MPID_COLL_NOSELECTION) - { - comm->coll_fns->Scan = MPIDO_Scan_simple; - MPIDI_UPDATE_COLLSEL_EXT_ALGO(MPIDO_CSWrapper_scan,"EXT:Scan:P2P:P2P",PAMI_XFER_SCAN); - } - } - } - TRACE_ERR("MPIDI_Comm_coll_envvars exit\n"); -} - - -/* Determine how many of each collective type this communicator supports */ -void MPIDI_Comm_coll_query(MPIR_Comm *comm) -{ - TRACE_ERR("MPIDI_Comm_coll_query enter\n"); - int rc = 0, i, j; - size_t num_algorithms[2]; - TRACE_ERR("Getting geometry from comm %p\n", comm); - pami_geometry_t *geom = comm->mpid.geometry;; - for(i = 0; i < PAMI_XFER_COUNT; i++) - { - if(i == PAMI_XFER_AMBROADCAST || i == PAMI_XFER_AMSCATTER || - i == PAMI_XFER_AMGATHER || i == PAMI_XFER_AMREDUCE) - continue; - rc = PAMI_Geometry_algorithms_num(geom, - i, - num_algorithms); - TRACE_ERR("Num algorithms of type %d: %zd %zd\n", i, num_algorithms[0], num_algorithms[1]); - if(rc != PAMI_SUCCESS) - { - fprintf(stderr,"PAMI_Geometry_algorithms_num returned %d for type %d\n", rc, i); - continue; - } - - comm->mpid.coll_count[i][0] = 0; - comm->mpid.coll_count[i][1] = 0; - if(num_algorithms[0] || num_algorithms[1]) - { - comm->mpid.coll_algorithm[i][0] = (pami_algorithm_t *) - MPL_malloc(sizeof(pami_algorithm_t) * num_algorithms[0]); - comm->mpid.coll_metadata[i][0] = (pami_metadata_t *) - MPL_malloc(sizeof(pami_metadata_t) * num_algorithms[0]); - comm->mpid.coll_algorithm[i][1] = (pami_algorithm_t *) - MPL_malloc(sizeof(pami_algorithm_t) * num_algorithms[1]); - comm->mpid.coll_metadata[i][1] = (pami_metadata_t *) - MPL_malloc(sizeof(pami_metadata_t) * num_algorithms[1]); - comm->mpid.coll_count[i][0] = num_algorithms[0]; - comm->mpid.coll_count[i][1] = num_algorithms[1]; - - /* Despite the bad name, this looks at algorithms associated with - * the geometry, NOT inherent physical properties of the geometry*/ - - /* BES TODO I am assuming all contexts have the same algorithms. Probably - * need to investigate that assumption - */ - rc = PAMI_Geometry_algorithms_query(geom, - i, - comm->mpid.coll_algorithm[i][0], - comm->mpid.coll_metadata[i][0], - num_algorithms[0], - comm->mpid.coll_algorithm[i][1], - comm->mpid.coll_metadata[i][1], - num_algorithms[1]); - if(rc != PAMI_SUCCESS) - { - fprintf(stderr,"PAMI_Geometry_algorithms_query returned %d for type %d\n", rc, i); - continue; - } - - if(MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_0 && comm->rank == 0) - { - for(j = 0; j < num_algorithms[0]; j++) - fprintf(stderr,"comm[%p] coll type %d (%s), algorithm %d 0: %s\n", comm, i, MPIDI_Coll_type_name(i), j, comm->mpid.coll_metadata[i][0][j].name); - for(j = 0; j < num_algorithms[1]; j++) - fprintf(stderr,"comm[%p] coll type %d (%s), algorithm %d 1: %s\n", comm, i, MPIDI_Coll_type_name(i), j, comm->mpid.coll_metadata[i][1][j].name); - if(i == PAMI_XFER_ALLGATHERV_INT || i == PAMI_XFER_ALLGATHER) - { - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_ALLREDUCE\n", comm, i, MPIDI_Coll_type_name(i)); - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_BCAST\n", comm, i, MPIDI_Coll_type_name(i)); - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_ALLTOALL\n", comm, i, MPIDI_Coll_type_name(i)); - } - if(i == PAMI_XFER_SCATTERV_INT) - { - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_BCAST\n", comm, i, MPIDI_Coll_type_name(i)); - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_ALLTOALLV\n", comm, i, MPIDI_Coll_type_name(i)); - } - if(i == PAMI_XFER_SCATTER) - { - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_BCAST\n", comm, i, MPIDI_Coll_type_name(i)); - } - if(i == PAMI_XFER_REDUCE) - { - fprintf(stderr,"comm[%p] coll type %d (%s), \"glue\" algorithm: GLUE_ALLREDUCE\n", comm, i, MPIDI_Coll_type_name(i)); - } - } - } - } - /* Determine if we have protocols for these maybe, rather than just setting them? */ - comm->coll_fns->Barrier = MPIDO_Barrier; - comm->coll_fns->Bcast = MPIDO_Bcast; - comm->coll_fns->Allreduce = MPIDO_Allreduce; - comm->coll_fns->Allgather = MPIDO_Allgather; - comm->coll_fns->Allgatherv = MPIDO_Allgatherv; - comm->coll_fns->Scatterv = MPIDO_Scatterv; - comm->coll_fns->Scatter = MPIDO_Scatter; - comm->coll_fns->Gather = MPIDO_Gather; - comm->coll_fns->Alltoallv = MPIDO_Alltoallv; - comm->coll_fns->Alltoall = MPIDO_Alltoall; - comm->coll_fns->Gatherv = MPIDO_Gatherv; - comm->coll_fns->Reduce = MPIDO_Reduce; - comm->coll_fns->Scan = MPIDO_Scan; - comm->coll_fns->Exscan = MPIDO_Exscan; - comm->coll_fns->Reduce_scatter_block = MPIDO_Reduce_scatter_block; - comm->coll_fns->Reduce_scatter = MPIDO_Reduce_scatter; - - /* MPI-3 Support, no optimized collectives hooked in yet */ - comm->coll_fns->Ibarrier_sched = MPIR_Ibarrier_intra; - comm->coll_fns->Ibcast_sched = MPIR_Ibcast_intra; - comm->coll_fns->Igather_sched = MPIR_Igather_intra; - comm->coll_fns->Igatherv_sched = MPIR_Igatherv; - comm->coll_fns->Iscatter_sched = MPIR_Iscatter_intra; - comm->coll_fns->Iscatterv_sched = MPIR_Iscatterv; - comm->coll_fns->Iallgather_sched = MPIR_Iallgather_intra; - comm->coll_fns->Iallgatherv_sched = MPIR_Iallgatherv_intra; - comm->coll_fns->Ialltoall_sched = MPIR_Ialltoall_intra; - comm->coll_fns->Ialltoallv_sched = MPIR_Ialltoallv_intra; - comm->coll_fns->Ialltoallw_sched = MPIR_Ialltoallw_intra; - comm->coll_fns->Iallreduce_sched = MPIR_Iallreduce_intra; - comm->coll_fns->Ireduce_sched = MPIR_Ireduce_intra; - comm->coll_fns->Ireduce_scatter_sched = MPIR_Ireduce_scatter_intra; - comm->coll_fns->Ireduce_scatter_block_sched = MPIR_Ireduce_scatter_block_intra; - comm->coll_fns->Iscan_sched = MPIR_Iscan_rec_dbl; - comm->coll_fns->Iexscan_sched = MPIR_Iexscan; - comm->coll_fns->Neighbor_allgather = MPIR_Neighbor_allgather_default; - comm->coll_fns->Neighbor_allgatherv = MPIR_Neighbor_allgatherv_default; - comm->coll_fns->Neighbor_alltoall = MPIR_Neighbor_alltoall_default; - comm->coll_fns->Neighbor_alltoallv = MPIR_Neighbor_alltoallv_default; - comm->coll_fns->Neighbor_alltoallw = MPIR_Neighbor_alltoallw_default; - comm->coll_fns->Ineighbor_allgather = MPIR_Ineighbor_allgather_default; - comm->coll_fns->Ineighbor_allgatherv = MPIR_Ineighbor_allgatherv_default; - comm->coll_fns->Ineighbor_alltoall = MPIR_Ineighbor_alltoall_default; - comm->coll_fns->Ineighbor_alltoallv = MPIR_Ineighbor_alltoallv_default; - comm->coll_fns->Ineighbor_alltoallw = MPIR_Ineighbor_alltoallw_default; - - /* MPI-3 Support, optimized collectives hooked in */ - comm->coll_fns->Ibarrier_req = MPIDO_Ibarrier; - comm->coll_fns->Ibcast_req = MPIDO_Ibcast; - comm->coll_fns->Iallgather_req = MPIDO_Iallgather; - comm->coll_fns->Iallgatherv_req = MPIDO_Iallgatherv; - comm->coll_fns->Iallreduce_req = MPIDO_Iallreduce; - comm->coll_fns->Ialltoall_req = MPIDO_Ialltoall; - comm->coll_fns->Ialltoallv_req = MPIDO_Ialltoallv; - comm->coll_fns->Ialltoallw_req = MPIDO_Ialltoallw; - comm->coll_fns->Iexscan_req = MPIDO_Iexscan; - comm->coll_fns->Igather_req = MPIDO_Igather; - comm->coll_fns->Igatherv_req = MPIDO_Igatherv; - comm->coll_fns->Ireduce_scatter_block_req = MPIDO_Ireduce_scatter_block; - comm->coll_fns->Ireduce_scatter_req = MPIDO_Ireduce_scatter; - comm->coll_fns->Ireduce_req = MPIDO_Ireduce; - comm->coll_fns->Iscan_req = MPIDO_Iscan; - comm->coll_fns->Iscatter_req = MPIDO_Iscatter; - comm->coll_fns->Iscatterv_req = MPIDO_Iscatterv; - - TRACE_ERR("MPIDI_Comm_coll_query exit\n"); -} - diff --git a/src/mpid/pamid/src/dyntask/Makefile.mk b/src/mpid/pamid/src/dyntask/Makefile.mk deleted file mode 100644 index 508de5d..0000000 --- a/src/mpid/pamid/src/dyntask/Makefile.mk +++ /dev/null @@ -1,32 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/dyntask/mpid_comm_spawn_multiple.c \ - src/mpid/pamid/src/dyntask/mpid_comm_disconnect.c \ - src/mpid/pamid/src/dyntask/mpid_port.c \ - src/mpid/pamid/src/dyntask/mpidi_pg.c \ - src/mpid/pamid/src/dyntask/mpidi_port.c - - -endif BUILD_PAMID diff --git a/src/mpid/pamid/src/dyntask/mpid_comm_disconnect.c b/src/mpid/pamid/src/dyntask/mpid_comm_disconnect.c deleted file mode 100644 index c573c82..0000000 --- a/src/mpid/pamid/src/dyntask/mpid_comm_disconnect.c +++ /dev/null @@ -1,541 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -#include "mpidimpl.h" - -#ifdef DYNAMIC_TASKING - -extern conn_info *_conn_info_list; - -#define DISCONNECT_LAPI_XFER_TIMEOUT 5*60*1000000 -#define TOTAL_AM 3 -#define FIRST_AM 0 -#define SECOND_AM 1 -#define LAST_AM 2 - -/* Returns time in micro seconds "double ticks" */ -#define CURTIME(ticks) { \ - struct timeval tp; \ - struct timezone tzp; \ - gettimeofday(&tp,&tzp); \ - ticks = (double) tp.tv_sec * 1000000 + (double) tp.tv_usec; \ -} - - -/* Used inside termination message send by smaller task to larger task in discon -nect */ -typedef struct { - long long tranid; - int whichAM; -}AM_struct2; - - -extern transactionID_struct *_transactionID_list; -void MPIDI_get_allremote_leaders(int *tid_arr, MPIR_Comm *comm_ptr); - -void MPIDI_send_AM_to_remote_leader_on_disconnect(int taskid, long long comm_cntr, int whichAM) -{ - pami_send_immediate_t xferP; - - int rc, current_val; - AM_struct2 AM_data; - pami_endpoint_t dest; - - AM_data.tranid = comm_cntr; - AM_data.whichAM = whichAM; - - memset(&xferP, 0, sizeof(pami_send_immediate_t)); - xferP.header.iov_base = (void*)&AM_data; - xferP.header.iov_len = sizeof(AM_struct2); - xferP.dispatch = (size_t)MPIDI_Protocols_Dyntask_disconnect; - - rc = PAMI_Endpoint_create(MPIDI_Client, taskid, 0, &dest); - TRACE_ERR("PAMI_Resume to taskid=%d\n", taskid); - PAMI_Resume(MPIDI_Context[0], - &dest, 1); - - if(rc != 0) - TRACE_ERR("PAMI_Endpoint_create failed\n"); - - xferP.dest = dest; - - rc = PAMI_Send_immediate(MPIDI_Context[0], &xferP); -} - -void MPIDI_Recvfrom_remote_world_disconnect(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - AM_struct2 *AM_data; - long long tranid; - int whichAM; - - AM_data = ((AM_struct2 *)_msginfo); - tranid = AM_data->tranid; - whichAM = AM_data->whichAM; - MPIDI_increment_AM_cntr_for_tranid(tranid, whichAM); - - TRACE_ERR("MPIDI_Recvfrom_remote_world_disconnect: invoked for tranid = %lld, whichAM = %d \n",tranid, whichAM); - - return; -} - - -/** - * Function to retreive the active message counter for a particular trasaction id. - * This function is used inside disconnect routine. - * whichAM = FIRST_AM/SECOND_AM/LAST_AM - */ -int MPIDI_get_AM_cntr_for_tranid(long long tranid, int whichAM) -{ - transactionID_struct *tridtmp; - - if(_transactionID_list == NULL) - TRACE_ERR("MPIDI_get_AM_cntr_for_tranid - _transactionID_list is NULL\n"); - - tridtmp = _transactionID_list; - - while(tridtmp != NULL) { - if(tridtmp->tranid == tranid) { - return tridtmp->cntr_for_AM[whichAM]; - } - tridtmp = tridtmp->next; - } - - return -1; -} - - -/** - * Function used to gurantee the delivery of LAPI active message at the - * destination. Called at the destination taskid and returns only whe - * the expected LAPI active message is recived. If the sequence number - * of the LAPI active message is LAST_AM, the other condition under which - * this function may exit is if DISCONNECT_LAPI_XFER_TIMEOUT happens. - */ -void MPIDI_wait_for_AM(long long tranid, int expected_AM, int whichAM) -{ - double starttime, currtime, elapsetime; - int rc, curr_AMcntr; - - rc = PAMI_Context_advance(MPIDI_Context[0], (size_t)100); - if(whichAM == LAST_AM) { - CURTIME(starttime) - do { - CURTIME(currtime) - elapsetime = currtime - starttime; - - rc = PAMI_Context_advance(MPIDI_Context[0], (size_t)100); - curr_AMcntr = MPIDI_get_AM_cntr_for_tranid(tranid, whichAM); - /*TRACE_ERR("_try_to_disconnect: Looping in timer for TranID %lld, whichAM %d expected_AM = %d, Current AM = %d\n",tranid,whichAM,expected_AM,curr_AMcntr); */ - }while(curr_AMcntr != expected_AM && elapsetime < DISCONNECT_LAPI_XFER_TIMEOUT); - } - else { - do { - rc = PAMI_Context_advance(MPIDI_Context[0], (size_t)100); - curr_AMcntr = MPIDI_get_AM_cntr_for_tranid(tranid, whichAM); - /*TRACE_ERR("_try_to_disconnect: Looping in timer for TranID %lld, whichAM %d expected_AM = %d, Current AM = %d\n",tranid,whichAM,expected_AM,curr_AMcntr);*/ - }while(curr_AMcntr != expected_AM); - } -} - -/* function to swap two integers. Used inside function _qsort_dyntask below */ -static void _swap_dyntask(int t[],int i,int j) -{ - int tmp; - - tmp = t[i]; - t[i] = t[j]; - t[j] = tmp; -} - -/* qsort sorting function which is used only in this file */ -static void _qsort_dyntask(int t[],int left,int right) -{ - int i,last; - - if(left >= right) return; - last = left; - for(i=left+1;i<=right;i++) - if(t[i] < t[left]) - _swap_dyntask(t,++last,i); - _swap_dyntask(t,left,last); - _qsort_dyntask(t,left,last-1); - _qsort_dyntask(t,last+1,right); -} - - - -/*@ - MPID_Comm_disconnect - Disconnect a communicator - - Arguments: -. comm_ptr - communicator - - Notes: - -.N Errors -.N MPI_SUCCESS -@*/ -int MPID_Comm_disconnect(MPIR_Comm *comm_ptr) -{ - int rc, i,j, k, ref_count,mpi_errno=0, probe_flag=0; - pami_task_t *local_list; - MPI_Status status; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - MPIDI_PG_t *pg; - int total_leaders=0, gsize; - pami_task_t *leader_tids; - int expected_firstAM=0, expected_secondAM=0, expected_lastAM=0; - MPIR_Comm *commworld_ptr; - MPIR_Group *group_ptr = NULL, *new_group_ptr = NULL; - MPID_VCR *glist; - MPIR_Comm *lcomm; - int *ranks; - int local_tasks=0, localtasks_in_remglist=0; - int jobIdSize=64; - char jobId[jobIdSize]; - int MY_TASKID = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval; - - /*if( (comm_ptr->comm_kind == MPIR_COMM_KIND__INTERCOMM) && (comm_ptr->mpid.world_ids != NULL)) { */ - if(comm_ptr->mpid.world_ids != NULL) { - rc = MPID_Iprobe(comm_ptr->rank, MPI_ANY_TAG, comm_ptr, MPIR_CONTEXT_INTER_PT2PT, &probe_flag, &status); - if(rc || probe_flag) { - TRACE_ERR("PENDING_PTP"); - exit(1); - } - - /* make commSubWorld */ - { - /* MPIR_Comm_get_ptr( MPI_COMM_WORLD, commworld_ptr ); */ - commworld_ptr = MPIR_Process.comm_world; - mpi_errno = MPIR_Comm_group_impl(commworld_ptr, &group_ptr); - if (mpi_errno) - { - TRACE_ERR("Error while creating group_ptr from MPI_COMM_WORLD in MPIDI_Comm_create_from_pami_geom\n"); - return PAMI_ERROR; - } - - - glist = commworld_ptr->vcr; - gsize = commworld_ptr->local_size; - for(i=0;i<comm_ptr->local_size;i++) { - for(j=0;j<gsize;j++) { - if(comm_ptr->local_vcr[i]->taskid == glist[j]->taskid) - local_tasks++; - } - } - - /** - * Tasks belonging to the same local world may also be part of - * the GROUPREMLIST, so these tasks will have to be use in addition - * to the tasks in GROUPLIST to construct lcomm - **/ - if(comm_ptr->comm_kind == MPIR_COMM_KIND__INTERCOMM) { - for(i=0;i<comm_ptr->remote_size;i++) { - for(j=0;j<gsize;j++) { - if(comm_ptr->vcr[i]->taskid == glist[j]->taskid) { - local_tasks++; - localtasks_in_remglist++; - } - } - } - } - k=0; - /* local_list = MPL_malloc(local_tasks*sizeof(pami_task_t)); */ - ranks = MPL_malloc(local_tasks*sizeof(int)); - - for(i=0;i<comm_ptr->local_size;i++) { - for(j=0;j<gsize;j++) { - if(comm_ptr->local_vcr[i]->taskid == glist[j]->taskid) - /* local_list[k] = glist[j]->taskid; */ - ranks[k++] = j; - } - } - if((comm_ptr->comm_kind == MPIR_COMM_KIND__INTERCOMM) && localtasks_in_remglist) { - for(i=0;i<comm_ptr->remote_size;i++) { - for(j=0;j<gsize;j++) { - if(comm_ptr->vcr[i]->taskid == glist[j]->taskid) - /* local_list[k] = glist[j]->taskid; */ - ranks[k++] = j; - } - } - /* Sort the local_list when there are localtasks_in_remglist */ -/* _qsort_dyntask(local_list, 0, local_tasks-1); */ - _qsort_dyntask(ranks, 0, local_tasks-1); - } - - /* Now we have all we need to create the new group. Create it */ - /* mpi_errno = MPIR_Group_incl_impl(group_ptr, local_tasks, ranks, &new_group_ptr); */ - mpi_errno = MPIR_Group_incl_impl(group_ptr, local_tasks, ranks, &new_group_ptr); - if (mpi_errno) - { - TRACE_ERR("Error while creating new_group_ptr from group_ptr in MPIDI_Comm_create_from_pami_geom\n"); - return PAMI_ERROR; - } - - /* Now create the communicator using the new_group_ptr */ - mpi_errno = MPIR_Comm_create_group(commworld_ptr, new_group_ptr, 0, &lcomm); - /* mpi_errno = MPIR_Comm_create_intra(commworld_ptr, new_group_ptr, &lcomm); */ - if (mpi_errno) - { - TRACE_ERR("Error while creating new_comm_ptr from group_ptr in MPIDI_Comm_create_from_pami_geom\n"); - return PAMI_ERROR; - } - -#if 0 - mpi_errno = MPIR_Comm_create(&lcomm); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Comm_create returned with mpi_errno=%d\n", mpi_errno); - } - - /* fill in all the fields of lcomm. */ - if(localtasks_in_remglist==0) { - lcomm->context_id = MPIR_CONTEXT_SET_FIELD(DYNAMIC_PROC, comm_ptr->recvcontext_id, 1); - lcomm->recvcontext_id = lcomm->context_id; - } else { - lcomm->context_id = MPIR_CONTEXT_SET_FIELD(DYNAMIC_PROC, comm_ptr->recvcontext_id, 1); - lcomm->recvcontext_id = MPIR_CONTEXT_SET_FIELD(DYNAMIC_PROC, comm_ptr->context_id, 1); - } - TRACE_ERR("lcomm->context_id =%d\n", lcomm->context_id); - - /* sanity: the INVALID context ID value could potentially conflict with the - * dynamic proccess space */ - MPIR_Assert(lcomm->context_id != MPIR_INVALID_CONTEXT_ID); - MPIR_Assert(lcomm->recvcontext_id != MPIR_INVALID_CONTEXT_ID); - - /* FIXME - we probably need a unique context_id. */ - - /* Fill in new intercomm */ - lcomm->comm_kind = MPIR_COMM_KIND__INTRACOMM; - lcomm->remote_size = lcomm->local_size = local_tasks; - - /* Set up VC reference table */ - mpi_errno = MPID_VCRT_Create(lcomm->remote_size, &lcomm->vcrt); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPID_VCRT_Create returned with mpi_errno=%d", mpi_errno); - } - MPID_VCRT_Add_ref(lcomm->vcrt); - mpi_errno = MPID_VCRT_Get_ptr(lcomm->vcrt, &lcomm->vcr); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPID_VCRT_Get_ptr returned with mpi_errno=%d", mpi_errno); - } - - for(i=0; i<local_tasks; i++) { - if(MY_TASKID == local_list[i]) lcomm->rank = i; - lcomm->vcr[i]->taskid = local_list[i]; - } -#endif - - } - - TRACE_ERR("subcomm for disconnect is established local_tasks=%d calling MPIR_Barrier_intra\n", local_tasks); - mpi_errno = MPIR_Barrier_intra(lcomm, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Barrier_intra returned with mpi_errno=%d\n", mpi_errno); - } - TRACE_ERR("after barrier in disconnect\n"); - - if(MY_TASKID != comm_ptr->mpid.local_leader) { - for(i=0; comm_ptr->mpid.world_ids[i] != -1; i++) { - ref_count = MPIDI_Decrement_ref_count(comm_ptr->mpid.world_ids[i]); - TRACE_ERR("ref_count=%d with world=%d comm_ptr=%x\n", ref_count, comm_ptr->mpid.world_ids[i], comm_ptr); - if(ref_count == -1) - TRACE_ERR("something is wrong\n"); - } - } - - if(MY_TASKID == comm_ptr->mpid.local_leader) { - PMI2_Job_GetId(jobId, jobIdSize); - for(i=0;comm_ptr->mpid.world_ids[i]!=-1;i++) { - if(atoi(jobId) != comm_ptr->mpid.world_ids[i]) - total_leaders++; - } - TRACE_ERR("total_leaders=%d\n", total_leaders); - leader_tids = MPL_malloc(total_leaders*sizeof(int)); - MPIDI_get_allremote_leaders(leader_tids, comm_ptr); - - { /* First Pair of Send / Recv -- All smaller task send to all larger tasks */ - for(i=0;i<total_leaders;i++) { - MPID_assert(leader_tids[i] != -1); - if(MY_TASKID < leader_tids[i]) { - TRACE_ERR("_try_to_disconnect: FIRST: comm_ptr->mpid.world_intercomm_cntr %lld, toTaskid %d\n",comm_ptr->mpid.world_intercomm_cntr,leader_tids[i]); - expected_firstAM++; - MPIDI_send_AM_to_remote_leader_on_disconnect(leader_tids[i], comm_ptr->mpid.world_intercomm_cntr, FIRST_AM); - } - else { - expected_secondAM++; - } - } - if(expected_secondAM) { - MPIDI_wait_for_AM(comm_ptr->mpid.world_intercomm_cntr, expected_secondAM, FIRST_AM); - } - } - - { /* Second Pair of Send / Recv -- All larger tasks send to all smaller tasks */ - for(i=0;i<total_leaders;i++) { - MPID_assert(leader_tids[i] != -1); - if(MY_TASKID > leader_tids[i]) { - TRACE_ERR("_try_to_disconnect: SECOND: comm_ptr->mpid.world_intercomm_cntr %lld, toTaskid %d\n",comm_ptr->mpid.world_intercomm_cntr,leader_tids[i]); - MPIDI_send_AM_to_remote_leader_on_disconnect(leader_tids[i], comm_ptr->mpid.world_intercomm_cntr, SECOND_AM); - } - } - if(expected_firstAM) { - MPIDI_wait_for_AM(comm_ptr->mpid.world_intercomm_cntr, expected_firstAM, SECOND_AM); - } - } - - for(i=0; comm_ptr->mpid.world_ids[i] != -1; i++) { - ref_count = MPIDI_Decrement_ref_count(comm_ptr->mpid.world_ids[i]); - TRACE_ERR("ref_count=%d with world=%d comm_ptr=%x\n", ref_count, comm_ptr->mpid.world_ids[i], comm_ptr); - if(ref_count == -1) - TRACE_ERR("something is wrong\n"); - } - - for(i=0;i<total_leaders;i++) { - MPID_assert(leader_tids[i] != -1); - if(MY_TASKID < leader_tids[i]) { - TRACE_ERR("_try_to_disconnect: LAST: comm_ptr->mpid.world_intercomm_cntr %lld, toTaskid %d\n",comm_ptr->mpid.world_intercomm_cntr,leader_tids[i]); - MPIDI_send_AM_to_remote_leader_on_disconnect(leader_tids[i], comm_ptr->mpid.world_intercomm_cntr, LAST_AM); - } - else { - expected_lastAM++; - } - } - if(expected_lastAM) { - MPIDI_wait_for_AM(comm_ptr->mpid.world_intercomm_cntr, expected_lastAM, - LAST_AM); - } - MPL_free(leader_tids); - } - - TRACE_ERR("_try_to_disconnect: Going inside final barrier for tranid %lld\n",comm_ptr->mpid.world_intercomm_cntr); - mpi_errno = MPIR_Barrier_intra(lcomm, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Barrier_intra returned with mpi_errno=%d\n", mpi_errno); - } - mpi_errno = MPIR_Comm_release(lcomm,0); - if (mpi_errno) TRACE_ERR("MPIR_Comm_release returned with mpi_errno=%d\n", mpi_errno); - - MPIDI_free_tranid_node(comm_ptr->mpid.world_intercomm_cntr); - mpi_errno = MPIR_Comm_release(comm_ptr,1); - if (mpi_errno) TRACE_ERR("MPIR_Comm_release returned with mpi_errno=%d\n", mpi_errno); - /* MPL_free(local_list); */ - MPL_free(ranks); - } - return mpi_errno; -} - - -int MPIDI_Decrement_ref_count(int wid) { - conn_info *tmp_node; - int ref_count=-1; - - tmp_node = _conn_info_list; - while(tmp_node != NULL) { - if(tmp_node->rem_world_id == wid) { - ref_count = --tmp_node->ref_count; - TRACE_ERR("decrement_ref_count: ref_count decremented to %d for remote world %d\n",ref_count,wid); - break; - } - tmp_node = tmp_node->next; - } - return ref_count; -} - -void MPIDI_get_allremote_leaders(int *tid_arr, MPIR_Comm *comm_ptr) -{ - conn_info *tmp_node; - int i,j,k,arr_len,gsize, found=0; - int leader1=-1, leader2=-1; - MPID_VCR *glist; - - for(i=0;comm_ptr->mpid.world_ids[i] != -1;i++) - { - TRACE_ERR("i=%d comm_ptr->mpid.world_ids[i]=%d\n", i, comm_ptr->mpid.world_ids[i]); - tmp_node = _conn_info_list; - found=0; - if(tmp_node==NULL) {TRACE_ERR("_conn_info_list is NULL\n");} - while(tmp_node != NULL) { - if(tmp_node->rem_world_id == comm_ptr->mpid.world_ids[i]) { - if(comm_ptr->comm_kind == MPIR_COMM_KIND__INTRACOMM) { - glist = comm_ptr->local_vcr; - gsize = comm_ptr->local_size; - } - else { - glist = comm_ptr->vcr; - gsize = comm_ptr->remote_size; - } - for(j=0;j<gsize;j++) { - for(k=0;tmp_node->rem_taskids[k]!=-1;k++) { - TRACE_ERR("j=%d k=%d glist[j]->taskid=%d tmp_node->rem_taskids[k]=%d\n", j, k,glist[j]->taskid, tmp_node->rem_taskids[k]); - if(glist[j]->taskid == tmp_node->rem_taskids[k]) { - leader1 = glist[j]->taskid; - found = 1; - break; - } - } - if(found) break; - } - /* - * There may be the case where my local_comm's GROUPLIST contains tasks - * fro remote world-x and GROUPREMLIST contains other remaining tasks of world-x - * If the smallest task of world-x is in my GROUPLIST then the above iteration - * will give the leader as smallest task from world-x in my GROUPREMLIST. - * But this will not be the correct leader_taskid. I should find the smallest task - * of world-x in my GROUPLIST and then see which of the two leaders is the - * smallest one. The smallest one is the one in which I am interested. - **/ - if(comm_ptr->comm_kind == MPIR_COMM_KIND__INTERCOMM) { - found=0; - glist = comm_ptr->local_vcr; - gsize = comm_ptr->local_size; - for(j=0;j<gsize;j++) { - for(k=0;tmp_node->rem_taskids[k]!=-1;k++) { - TRACE_ERR("j=%d k=%d glist[j]->taskid=%d tmp_node->rem_taskids[k]=%d\n", j, k, glist[j]->taskid, tmp_node->rem_taskids[k]); - if(glist[j]->taskid == tmp_node->rem_taskids[k]) { - leader2 = glist[j]->taskid; - found = 1; - break; - } - } - if(found) break; - } - } - if(found) { - break; - } - } else {TRACE_ERR("world id is different tmp_node->rem_world_id =%d comm_ptr=%x comm_ptr->mpid.world_ids[i]=%d\n", tmp_node->rem_world_id, comm_ptr, comm_ptr->mpid.world_ids[i]);} - tmp_node = tmp_node->next; - } - - TRACE_ERR("comm_ptr=%x leader1=%d leader2=%d\n", comm_ptr, leader1, leader2); - if(leader1 == -1) - *(tid_arr+i) = leader2; - else if(leader2 == -1) - *(tid_arr+i) = leader1; - else - *(tid_arr+i) = leader1 < leader2 ? leader1 : leader2; - } -} - -#endif diff --git a/src/mpid/pamid/src/dyntask/mpid_comm_spawn_multiple.c b/src/mpid/pamid/src/dyntask/mpid_comm_spawn_multiple.c deleted file mode 100644 index 6e3b21c..0000000 --- a/src/mpid/pamid/src/dyntask/mpid_comm_spawn_multiple.c +++ /dev/null @@ -1,414 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; -*- */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" -#ifdef USE_PMI2_API -#include "pmi2.h" -#else -#include "pmi.h" -#endif - -#ifdef DYNAMIC_TASKING - -extern mpidi_dynamic_tasking; - -/* Define the name of the kvs key used to provide the port name to the - children */ -#define MPIDI_PARENT_PORT_KVSKEY "PARENT_ROOT_PORT_NAME" - -/* FIXME: We can avoid these two routines if we define PMI as using - MPI info values */ -/* Turn a SINGLE MPI_Info into an array of PMI_keyvals (return the pointer - to the array of PMI keyvals) */ -static int MPIDI_mpi_to_pmi_keyvals( MPIR_Info *info_ptr, PMI_keyval_t **kv_ptr, int *nkeys_ptr ) -{ - char key[MPI_MAX_INFO_KEY]; - PMI_keyval_t *kv = 0; - int i, nkeys = 0, vallen, flag, mpi_errno=MPI_SUCCESS; - - if (!info_ptr || info_ptr->handle == MPI_INFO_NULL) { - goto fn_exit; - } - - MPIR_Info_get_nkeys_impl( info_ptr, &nkeys ); - if (nkeys == 0) { - goto fn_exit; - } - kv = (PMI_keyval_t *)MPL_malloc( nkeys * sizeof(PMI_keyval_t) ); - - for (i=0; i<nkeys; i++) { - mpi_errno = MPIR_Info_get_nthkey_impl( info_ptr, i, key ); - if (mpi_errno) - TRACE_ERR("MPIR_Info_get_nthkey_impl returned with mpi_errno=%d\n", mpi_errno); - MPIR_Info_get_valuelen_impl( info_ptr, key, &vallen, &flag ); - - kv[i].key = MPL_strdup(key); - kv[i].val = MPL_malloc( vallen + 1 ); - MPIR_Info_get_impl( info_ptr, key, vallen+1, kv[i].val, &flag ); - TRACE_OUT("key: <%s>, value: <%s>\n", kv[i].key, kv[i].val); - } - - fn_fail: - fn_exit: - *kv_ptr = kv; - *nkeys_ptr = nkeys; - return mpi_errno; -} - - -/* Free the entire array of PMI keyvals */ -static void MPIDI_free_pmi_keyvals(PMI_keyval_t **kv, int size, int *counts) -{ - int i,j; - - for (i=0; i<size; i++) - { - for (j=0; j<counts[i]; j++) - { - if (kv[i][j].key != NULL) - MPL_free((char *)kv[i][j].key); - if (kv[i][j].val != NULL) - MPL_free(kv[i][j].val); - } - if (kv[i] != NULL) - { - MPL_free(kv[i]); - } - } -} - -/*@ - MPID_Comm_spawn_multiple - - - Input Arguments: -+ int count - count -. char *array_of_commands[] - commands -. char* *array_of_argv[] - arguments -. int array_of_maxprocs[] - maxprocs -. MPI_Info array_of_info[] - infos -. int root - root -- MPI_Comm comm - communicator - - Output Arguments: -+ MPI_Comm *intercomm - intercommunicator -- int array_of_errcodes[] - error codes - - Notes: - -.N Errors -.N MPI_SUCCESS -@*/ -#undef FUNCNAME -#define FUNCNAME MPID_Comm_spawn_multiple -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_Comm_spawn_multiple(int count, char *array_of_commands[], - char ** array_of_argv[], const int array_of_maxprocs[], - MPIR_Info * array_of_info_ptrs[], int root, - MPIR_Comm * comm_ptr, MPIR_Comm ** intercomm, - int array_of_errcodes[]) -{ - int mpi_errno = MPI_SUCCESS; - - if(mpidi_dynamic_tasking == 0) { - fprintf(stderr, "Received spawn request for non-dynamic jobs\n"); - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_SPAWN, - return mpi_errno, "**spawn"); - } - - /* We allow an empty implementation of this function to - simplify building MPICH on systems that have difficulty - supporing process creation */ - mpi_errno = MPIDI_Comm_spawn_multiple(count, array_of_commands, - array_of_argv, array_of_maxprocs, - array_of_info_ptrs, - root, comm_ptr, intercomm, - array_of_errcodes); - return mpi_errno; -} - - -/* - * MPIDI_Comm_spawn_multiple() - */ -int MPIDI_Comm_spawn_multiple(int count, char **commands, - char ***argvs, int *maxprocs, - MPIR_Info **info_ptrs, int root, - MPIR_Comm *comm_ptr, MPIR_Comm - **intercomm, int *errcodes) -{ - char port_name[MPI_MAX_PORT_NAME]; - char jobId[64]; - char ctxid_str[16]; - int jobIdSize = 64; - int len=0; - int *info_keyval_sizes=0, i, mpi_errno=MPI_SUCCESS; - PMI_keyval_t **info_keyval_vectors=0, preput_keyval_vector; - int *pmi_errcodes = 0, pmi_errno=0; - int total_num_processes, should_accept = 1; - MPIR_Info tmp_info_ptr; - char *tmp; - int tmp_ret = 0; - - if (comm_ptr->rank == root) { - /* create an array for the pmi error codes */ - total_num_processes = 0; - for (i=0; i<count; i++) { - total_num_processes += maxprocs[i]; - } - pmi_errcodes = (int*)MPL_malloc(sizeof(int) * total_num_processes); - - /* initialize them to 0 */ - for (i=0; i<total_num_processes; i++) - pmi_errcodes[i] = 0; - - /* Open a port for the spawned processes to connect to */ - /* FIXME: info may be needed for port name */ - mpi_errno = MPID_Open_port(NULL, port_name); - TRACE_ERR("mpi_errno from MPID_Open_port=%d\n", mpi_errno); - - /* Spawn the processes */ -#ifdef USE_PMI2_API - MPIR_Assert(count > 0); - { - int *argcs = MPL_malloc(count*sizeof(int)); - struct MPIR_Info preput; - struct MPIR_Info *preput_p[2] = { &preput, &tmp_info_ptr }; - - MPIR_Assert(argcs); - - info_keyval_sizes = MPL_malloc(count * sizeof(int)); - - /* FIXME cheating on constness */ - preput.key = (char *)MPIDI_PARENT_PORT_KVSKEY; - preput.value = port_name; - preput.next = &tmp_info_ptr; - - tmp_info_ptr.key = "COMMCTX"; - len=sprintf(ctxid_str, "%d", comm_ptr->context_id); - TRACE_ERR("COMMCTX=%d\n", comm_ptr->context_id); - ctxid_str[len]='\0'; - tmp_info_ptr.value = ctxid_str; - tmp_info_ptr.next = NULL; - - /* compute argcs array */ - for (i = 0; i < count; ++i) { - argcs[i] = 0; - if (argvs != NULL && argvs[i] != NULL) { - while (argvs[i][argcs[i]]) { - ++argcs[i]; - } - } - } - - /*MPIU_THREAD_CS_ENTER(PMI,);*/ - /* release the global CS for spawn PMI calls */ - MPIU_THREAD_CS_EXIT(ALLFUNC,); - pmi_errno = PMI2_Job_Spawn(count, (const char **)commands, - argcs, (const char ***)argvs, - maxprocs, - info_keyval_sizes, (const MPIR_Info **)info_ptrs, - 2, (const struct MPIR_Info **)preput_p, - jobId, jobIdSize, - pmi_errcodes); - TRACE_ERR("after PMI2_Job_Spawn - pmi_errno=%d jobId=%s\n", pmi_errno, jobId); - MPIU_THREAD_CS_ENTER(ALLFUNC,); - - tmp=MPL_strdup(jobId); - tmp_ret = atoi(strtok(tmp, ";")); - - if( (pmi_errno == PMI2_SUCCESS) && (tmp_ret != -1) ) { - pami_task_t leader_taskid = atoi(strtok(NULL, ";")); - pami_endpoint_t ldest; - - PAMI_Endpoint_create(MPIDI_Client, leader_taskid, 0, &ldest); - TRACE_ERR("PAMI_Resume to taskid=%d\n", leader_taskid); - PAMI_Resume(MPIDI_Context[0], &ldest, 1); - } - - MPL_free(tmp); - - MPL_free(argcs); - if (pmi_errno != PMI2_SUCCESS) { - TRACE_ERR("PMI2_Job_Spawn returned with pmi_errno=%d\n", pmi_errno); - } - } -#else - /* FIXME: This is *really* awkward. We should either - Fix on MPI-style info data structures for PMI (avoid unnecessary - duplication) or add an MPIU_Info_getall(...) that creates - the necessary arrays of key/value pairs */ - - /* convert the infos into PMI keyvals */ - info_keyval_sizes = (int *) MPL_malloc(count * sizeof(int)); - info_keyval_vectors = - (PMI_keyval_t**) MPL_malloc(count * sizeof(PMI_keyval_t*)); - - if (!info_ptrs) { - for (i=0; i<count; i++) { - info_keyval_vectors[i] = 0; - info_keyval_sizes[i] = 0; - } - } - else { - for (i=0; i<count; i++) { - mpi_errno = MPIDI_mpi_to_pmi_keyvals( info_ptrs[i], - &info_keyval_vectors[i], - &info_keyval_sizes[i] ); - if (mpi_errno) { TRACE_ERR("MPIDI_mpi_to_pmi_keyvals returned with mpi_errno=%d\n", mpi_errno); } - } - } - - preput_keyval_vector.key = MPIDI_PARENT_PORT_KVSKEY; - preput_keyval_vector.val = port_name; - - pmi_errno = PMI_Spawn_multiple(count, (const char **) - commands, - (const char ***) argvs, - maxprocs, info_keyval_sizes, - (const PMI_keyval_t **) - info_keyval_vectors, 1, - &preput_keyval_vector, - pmi_errcodes); - TRACE_ERR("pmi_errno from PMI_Spawn_multiple=%d\n", pmi_errno); -#endif - - if (errcodes != MPI_ERRCODES_IGNORE) { - for (i=0; i<total_num_processes; i++) { - /* FIXME: translate the pmi error codes here */ - errcodes[i] = pmi_errcodes[0]; - /* We want to accept if any of the spawns succeeded. - Alternatively, this is the same as we want to NOT accept if - all of them failed. should_accept = NAND(e_0, ..., e_n) - Remember, success equals false (0). */ - should_accept = should_accept && errcodes[i]; - } - should_accept = !should_accept; /* the `N' in NAND */ - } - -#ifdef USE_PMI2_API - if( (pmi_errno == PMI2_SUCCESS) && (tmp_ret == -1) ) -#else - if( (pmi_errno == PMI_SUCCESS) && (tmp_ret == -1) ) -#endif - should_accept = 0; - } - - if (errcodes != MPI_ERRCODES_IGNORE) { - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - mpi_errno = MPIR_Bcast_impl(&should_accept, 1, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_impl returned with mpi_errno=%d\n", mpi_errno); - - mpi_errno = MPIR_Bcast_impl(&pmi_errno, 1, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_impl returned with mpi_errno=%d\n", mpi_errno); - - mpi_errno = MPIR_Bcast_impl(&total_num_processes, 1, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_impl returned with mpi_errno=%d\n", mpi_errno); - - mpi_errno = MPIR_Bcast_impl(errcodes, total_num_processes, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_impl returned with mpi_errno=%d\n", mpi_errno); - } - - if (should_accept) { - mpi_errno = MPID_Comm_accept(port_name, NULL, root, comm_ptr, intercomm); - TRACE_ERR("mpi_errno from MPID_Comm_accept=%d\n", mpi_errno); - } else { - if( (pmi_errno == PMI2_SUCCESS) && (errcodes[0] != 0) ) { - MPIR_Comm_create(intercomm); - } - } - - if (comm_ptr->rank == root) { - /* Close the port opened for the spawned processes to connect to */ - mpi_errno = MPID_Close_port(port_name); - /* --BEGIN ERROR HANDLING-- */ - if (mpi_errno != MPI_SUCCESS) - TRACE_ERR("MPID_Close_port returned with mpi_errno=%d\n", mpi_errno); - /* --END ERROR HANDLING-- */ - } - - if(pmi_errno) { - mpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_FATAL, __FILE__, __LINE__, MPI_ERR_SPAWN, - "**mpi_comm_spawn", 0); - } - - fn_exit: - if (info_keyval_vectors) { - MPIDI_free_pmi_keyvals(info_keyval_vectors, count, info_keyval_sizes); - MPL_free(info_keyval_vectors); - } - if (info_keyval_sizes) { - MPL_free(info_keyval_sizes); - } - if (pmi_errcodes) { - MPL_free(pmi_errcodes); - } - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -/* This function is used only with mpid_init to set up the parent communicator - if there is one. The routine should be in this file because the parent - port name is setup with the "preput" arguments to PMI_Spawn_multiple */ -static char *parent_port_name = 0; /* Name of parent port if this - process was spawned (and is root - of comm world) or null */ -#undef FUNCNAME -#define FUNCNAME MPIDI_GetParentPort -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIDI_GetParentPort(char ** parent_port) -{ - int mpi_errno = MPI_SUCCESS; - int pmi_errno; - char val[MPIDI_MAX_KVS_VALUE_LEN]; - - if (parent_port_name == NULL) - { - char *kvsname = NULL; - /* We can always use PMI_KVS_Get on our own process group */ - MPIDI_PG_GetConnKVSname( &kvsname ); -#ifdef USE_PMI2_API - { - int vallen = 0; - pmi_errno = PMI2_KVS_Get(kvsname, PMI2_ID_NULL, MPIDI_PARENT_PORT_KVSKEY, val, sizeof(val), &vallen); - TRACE_ERR("PMI2_KVS_Get - val=%s\n", val); - if (pmi_errno) - TRACE_ERR("PMI2_KVS_Get returned with pmi_errno=%d\n", pmi_errno); - } -#else - /*MPIU_THREAD_CS_ENTER(PMI,);*/ - pmi_errno = PMI_KVS_Get( kvsname, MPIDI_PARENT_PORT_KVSKEY, val, sizeof(val)); -/* MPIU_THREAD_CS_EXIT(PMI,);*/ - if (pmi_errno) { - mpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_FATAL, FCNAME, __LINE__, MPI_ERR_OTHER, "**pmi_kvsget", "**pmi_kvsget %d", pmi_errno); - goto fn_exit; - } -#endif - parent_port_name = MPL_strdup(val); - } - - *parent_port = parent_port_name; - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -void MPIDI_FreeParentPort(void) -{ - if (parent_port_name) { - MPL_free( parent_port_name ); - parent_port_name = 0; - } -} - - -#endif diff --git a/src/mpid/pamid/src/dyntask/mpid_port.c b/src/mpid/pamid/src/dyntask/mpid_port.c deleted file mode 100644 index 7f115d8..0000000 --- a/src/mpid/pamid/src/dyntask/mpid_port.c +++ /dev/null @@ -1,317 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; -*- */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" -#include "netdb.h" -#include <net/if.h> - - -#ifdef DYNAMIC_TASKING - -#define MAX_HOST_DESCRIPTION_LEN 128 -#define NUM_IFREQS 10 -#define MPI_MAX_TASKID_NAME 8 -#define MPIDI_TASKID_TAG_KEY "taskid" - -extern mpidi_dynamic_tasking; - -static int MPIDI_Open_port(MPIR_Info *, char *); -static int MPIDI_Close_port(const char *); - - -/* Define the functions that are used to implement the port - * operations */ -static MPIDI_PortFns portFns = { MPIDI_Open_port, - MPIDI_Close_port, - MPIDI_Comm_accept, - MPIDI_Comm_connect }; - -/*@ - MPID_Open_port - Open an MPI Port - - Input Arguments: -. MPI_Info info - info - - Output Arguments: -. char *port_name - port name - - Notes: - - -.N Errors -.N MPI_SUCCESS -.N MPI_ERR_OTHER -@*/ -int MPID_Open_port(MPIR_Info *info_ptr, char *port_name) -{ - int mpi_errno=MPI_SUCCESS; - - if (portFns.OpenPort) { - mpi_errno = portFns.OpenPort( info_ptr, port_name ); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("OpenPort returned with mpi_errno=%d\n", mpi_errno); - } - } - - fn_fail: - return mpi_errno; -} - - - -/*@ - MPID_Close_port - Close port - - Input Parameter: -. port_name - Name of MPI port to close - - Notes: - -.N Errors -.N MPI_SUCCESS -.N MPI_ERR_OTHER - -@*/ -int MPID_Close_port(const char *port_name) -{ - int mpi_errno=MPI_SUCCESS; - - if (portFns.ClosePort) { - mpi_errno = portFns.ClosePort( port_name ); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("ClosePort returned with mpi_errno=%d\n", mpi_errno); - } - } - - fn_fail: - return mpi_errno; -} - -#undef FUNCNAME -#define FUNCNAME MPID_Comm_accept -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_Comm_accept(const char * port_name, MPIR_Info * info, int root, - MPIR_Comm * comm, MPIR_Comm ** newcomm_ptr) -{ - int mpi_errno = MPI_SUCCESS; - - if(mpidi_dynamic_tasking == 0) { - fprintf(stderr, "Dynamic tasking API is called on non-dynamic jobs\n"); - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_SPAWN, - return mpi_errno, "**spawn"); - - } - - if (portFns.CommAccept) { - mpi_errno = portFns.CommAccept( port_name, info, root, comm, - newcomm_ptr ); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("CommAccept returned with mpi_errno=%d\n", mpi_errno); - } - } - - fn_fail: - return mpi_errno; -} - -#undef FUNCNAME -#define FUNCNAME MPID_Comm_connect -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_Comm_connect(const char * port_name, MPIR_Info * info, int root, - MPIR_Comm * comm, MPIR_Comm ** newcomm_ptr) -{ - int mpi_errno=MPI_SUCCESS; - - if(mpidi_dynamic_tasking == 0) { - fprintf(stderr, "Dynamic tasking API is called on non-dynamic jobs\n"); - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_SPAWN, - return mpi_errno, "**spawn"); - - } - - if (portFns.CommConnect) { - mpi_errno = portFns.CommConnect( port_name, info, root, comm, - newcomm_ptr ); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("CommConnect returned with mpi_errno=%d\n", mpi_errno); - } - } - - fn_fail: - return mpi_errno; -} - - -/* - * Here are the routines that provide some of the default implementations - * for the Port routines. - * - * MPIDI_Open_port - creates a port "name" that includes a tag value that - * is used to separate different MPI Port values. That tag value is - * extracted with MPIDI_GetTagFromPort - * MPIDI_GetTagFromPort - Routine to return the tag associated with a port. - * - * The port_name_tag is used in the connect and accept messages that - * are used in the connect/accept protocol. - */ - -#define MPIDI_PORT_NAME_TAG_KEY "tag" -#define MPIDI_TASKID_TAG_KEY "taskid" - -/* Though the port_name_tag_mask itself is an int, we can only have as - * many tags as the context_id space can support. */ -static int port_name_tag_mask[MPIR_MAX_CONTEXT_MASK] = { 0 }; - -static int MPIDI_get_port_name_tag(int * port_name_tag) -{ - int i, j; - int mpi_errno = MPI_SUCCESS; - - - for (i = 0; i < MPIR_MAX_CONTEXT_MASK; i++) - if (port_name_tag_mask[i] != ~0) - break; - - TRACE_ERR("MPIDI_get_port_name_tag - i=%d MPIR_MAX_CONTEXT_MASK=%d", i, MPIR_MAX_CONTEXT_MASK); - if (i < MPIR_MAX_CONTEXT_MASK) { - /* Found a free tag. port_name_tag_mask[i] is not fully used - * up. */ - - /* OR the mask value with powers of two. If the OR value is - * the same as the original value, then it means that the - * OR'ed bit was originally 1 (used); otherwise, it was - * originally 0 (free). */ - for (j = 0; j < (8 * sizeof(int)); j++) { - if ((port_name_tag_mask[i] | (1 << ((8 * sizeof(int)) - j - 1))) != - port_name_tag_mask[i]) { - /* Mark the appropriate bit as used and return that */ - port_name_tag_mask[i] |= (1 << ((8 * sizeof(int)) - j - 1)); - *port_name_tag = ((i * 8 * sizeof(int)) + j); - goto fn_exit; - } - } - } - else { - goto fn_fail; - } - -fn_exit: - return mpi_errno; - -fn_fail: - /* Everything is used up */ - *port_name_tag = -1; - mpi_errno = MPI_ERR_OTHER; - goto fn_exit; -} - -static void MPIDI_free_port_name_tag(int tag) -{ - int index, rem_tag; - - index = tag / (sizeof(int) * 8); - rem_tag = tag - (index * sizeof(int) * 8); - - port_name_tag_mask[index] &= ~(1 << ((8 * sizeof(int)) - 1 - rem_tag)); -} - - -/* - * MPIDI_Open_port() - */ -static int MPIDI_Open_port(MPIR_Info *info_ptr, char *port_name) -{ - int mpi_errno = MPI_SUCCESS; - int str_errno = MPL_STR_SUCCESS; - int len; - int port_name_tag = 0; /* this tag is added to the business card, - which is then returned as the port name */ - int taskid_tag; - int myRank = MPIR_Process.comm_world->rank; - - mpi_errno = MPIDI_get_port_name_tag(&port_name_tag); - TRACE_ERR("MPIDI_get_port_name_tag - port_name_tag=%d mpi_errno=%d\n", port_name_tag, mpi_errno); - - len = MPI_MAX_PORT_NAME; - str_errno = MPL_str_add_int_arg(&port_name, &len, - MPIDI_PORT_NAME_TAG_KEY, port_name_tag); - /*len = MPI_MAX_TASKID_NAME;*/ - taskid_tag = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval; - str_errno = MPL_str_add_int_arg(&port_name, &len, - MPIDI_TASKID_TAG_KEY, taskid_tag); - TRACE_ERR("MPL_str_add_int_arg - port_name=%s str_errno=%d\n", port_name, str_errno); - - /* This works because Get_business_card uses the same MPL_str_xxx - functions as above to add the business card to the input string */ - /* FIXME: We should instead ask the mpid_pg routines to give us - a connection string. There may need to be a separate step to - restrict us to a connection information that is only valid for - connections between processes that are started separately (e.g., - may not use shared memory). We may need a channel-specific - function to create an exportable connection string. */ - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -/* - * MPIDI_Close_port() - */ -static int MPIDI_Close_port(const char *port_name) -{ - int mpi_errno = MPI_SUCCESS; - int port_name_tag; - - mpi_errno = MPIDI_GetTagFromPort(port_name, &port_name_tag); - - MPIDI_free_port_name_tag(port_name_tag); - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -/* - * The connect and accept routines use this routine to get the port tag - * from the port name. - */ -int MPIDI_GetTagFromPort( const char *port_name, int *port_name_tag ) -{ - int mpi_errno = MPI_SUCCESS; - int str_errno = MPL_STR_SUCCESS; - - str_errno = MPL_str_get_int_arg(port_name, MPIDI_PORT_NAME_TAG_KEY, - port_name_tag); - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -/* - * The connect and accept routines use this routine to get the port tag - * from the port name. - */ -int MPIDI_GetTaskidFromPort( const char *port_name, int *taskid_tag ) -{ - int mpi_errno = MPI_SUCCESS; - int str_errno = MPL_STR_SUCCESS; - - str_errno = MPL_str_get_int_arg(port_name, MPIDI_TASKID_TAG_KEY, - taskid_tag); - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} -#endif diff --git a/src/mpid/pamid/src/dyntask/mpidi_pg.c b/src/mpid/pamid/src/dyntask/mpidi_pg.c deleted file mode 100644 index a344b6a..0000000 --- a/src/mpid/pamid/src/dyntask/mpidi_pg.c +++ /dev/null @@ -1,1063 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; -*- */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ -#include <mpidimpl.h> -#ifdef USE_PMI2_API -#include "pmi2.h" -#else -#include "pmi.h" -#endif - -#ifdef DYNAMIC_TASKING - -extern int mpidi_dynamic_tasking; -int mpidi_sync_done=0; - - -#define MAX_JOBID_LEN 1024 - -/* FIXME: These routines need a description. What is their purpose? Who - calls them and why? What does each one do? -*/ -static MPIDI_PG_t * MPIDI_PG_list = NULL; -static MPIDI_PG_t * MPIDI_PG_iterator_next = NULL; -static MPIDI_PG_Compare_ids_fn_t MPIDI_PG_Compare_ids_fn; -static MPIDI_PG_Destroy_fn_t MPIDI_PG_Destroy_fn; - -/* Set verbose to 1 to record changes to the process group structure. */ -static int verbose = 0; - -/* Key track of the process group corresponding to the MPI_COMM_WORLD - of this process */ -MPIDI_PG_t *pg_world = NULL; - -#define MPIDI_MAX_KVS_KEY_LEN 256 - -extern conn_info *_conn_info_list; - -int MPIDI_PG_Init(int *argc_p, char ***argv_p, - MPIDI_PG_Compare_ids_fn_t compare_ids_fn, - MPIDI_PG_Destroy_fn_t destroy_fn) -{ - int mpi_errno = MPI_SUCCESS; - char *p; - - MPIDI_PG_Compare_ids_fn = compare_ids_fn; - MPIDI_PG_Destroy_fn = destroy_fn; - - /* Check for debugging options. We use MPICHD_DBG and -mpichd-dbg - to avoid confusion with the code in src/util/dbg/dbg_printf.c */ - p = getenv( "MPICHD_DBG_PG" ); - if (p && ( strcmp( p, "YES" ) == 0 || strcmp( p, "yes" ) == 0) ) - verbose = 1; - if (argc_p && argv_p) { - int argc = *argc_p, i; - char **argv = *argv_p; - /* applied patch from Juha Jeronen, req #3920 */ - for (i=1; i<argc && argv[i]; i++) { - if (strcmp( "-mpichd-dbg-pg", argv[i] ) == 0) { - int j; - verbose = 1; - for (j=i; j<argc-1; j++) { - argv[j] = argv[j+1]; - } - argv[argc-1] = NULL; - *argc_p = argc - 1; - break; - } - } - } - - return mpi_errno; -} - -void *mpidi_finalize_req(void *arg) { - PMI2_Finalize(); - mpidi_sync_done=1; -} - -/*@ - MPIDI_PG_Finalize - Finalize the process groups, including freeing all - process group structures - @*/ -int MPIDI_PG_Finalize(void) -{ - int mpi_errno = MPI_SUCCESS; - conn_info *conn_node; - int my_max_worldid, world_max_worldid; - int wid_bit_array_size=0, wid; - unsigned char *wid_bit_array=NULL, *root_wid_barray=NULL; - MPIDI_PG_t *pg, *pgNext; - char key[PMI2_MAX_KEYLEN]; - char value[PMI2_MAX_VALLEN]; - pthread_t finalize_req_thread; - - /* Print the state of the process groups */ - if (verbose) { - MPIU_PG_Printall( stdout ); - } - - /* FIXME - straighten out the use of PMI_Finalize - no use after - PG_Finalize */ - conn_node = _conn_info_list; - my_max_worldid = -1; - - while(NULL != conn_node) { - if(conn_node->rem_world_id>my_max_worldid && conn_node->ref_count>0) { - TRACE_ERR("conn_node->rem_world_id=%d conn_node->ref_count=%d\n", conn_node->rem_world_id, conn_node->ref_count); - my_max_worldid = conn_node->rem_world_id; - } else { - TRACE_ERR("conn_node->rem_world_id=%d conn_node->ref_count=%d\n", conn_node->rem_world_id, conn_node->ref_count); - } - conn_node = conn_node->next; - } - MPIR_Allreduce_impl( &my_max_worldid, &world_max_worldid, 1, MPI_INT, MPI_MAX, MPIR_Process.comm_world, &mpi_errno); - - /* create bit array of size = world_max_worldid + 1 - * We add 1 to world_max_worldid because suppose my world - * is only connected to world_id 0 then world_max_worldid=0 - * and if we do not add 1, then size of bite array will be 0. - * Also suppose in my world world_max_worldid is 8. Then if we - * dont add 1, then the bit array will be size 1 byte, and when - * we try to set bit in position 8, we will get segfault. - */ - TRACE_ERR("my_max_worldid=%d world_max_worldid=%d\n", my_max_worldid, world_max_worldid); - if(world_max_worldid != -1) { - world_max_worldid++; - wid_bit_array_size = (world_max_worldid + CHAR_BIT -1) / CHAR_BIT; - wid_bit_array = MPL_malloc(wid_bit_array_size*sizeof(unsigned char)); - memset(wid_bit_array, 0, wid_bit_array_size*sizeof(unsigned char)); - root_wid_barray = MPL_malloc(wid_bit_array_size*sizeof(unsigned char)); - - memset(root_wid_barray, 0, wid_bit_array_size*sizeof(unsigned char)); - conn_node = _conn_info_list; - while(NULL != conn_node) { - if(conn_node->ref_count >0) { - wid = conn_node->rem_world_id; - wid_bit_array[wid/CHAR_BIT] |= 1 << (wid%CHAR_BIT); - TRACE_ERR("wid=%d wid_bit_array[%d]=%x\n", wid, wid/CHAR_BIT, 1 << (wid%CHAR_BIT)); - } - conn_node = conn_node->next; - - } - /* Let root of my world know about this bit array */ - MPIR_Reduce_impl(wid_bit_array,root_wid_barray,wid_bit_array_size, - MPI_UNSIGNED_CHAR,MPI_BOR,0,MPIR_Process.comm_world,&mpi_errno); - - MPL_free(wid_bit_array); - } - - if(MPIR_Process.comm_world->rank == 0) { - - MPL_snprintf(key, PMI2_MAX_KEYLEN-1, "%s", "ROOTWIDARRAY"); - MPL_snprintf(value, PMI2_MAX_VALLEN-1, "%s", root_wid_barray); - TRACE_ERR("root_wid_barray=%s\n", value); - mpi_errno = PMI2_KVS_Put(key, value); - TRACE_ERR("PMI2_KVS_Put returned with mpi_errno=%d\n", mpi_errno); - - MPL_snprintf(key, PMI2_MAX_KEYLEN-1, "%s", "WIDBITARRAYSZ"); - MPL_snprintf(value, PMI2_MAX_VALLEN-1, "%x", wid_bit_array_size); - key[strlen(key)+1]='\0'; - value[strlen(value)+1]='\0'; - mpi_errno = PMI2_KVS_Put(key, value); - TRACE_ERR("PMI2_KVS_Put returned with mpi_errno=%d\n", mpi_errno); - - } - mpi_errno = PMI2_KVS_Fence(); - TRACE_ERR("PMI2_KVS_Fence returned with mpi_errno=%d\n", mpi_errno); - - MPL_free(root_wid_barray); /* root_wid_barray is now NULL for non-root */ - -#if 0 - pthread_create(&finalize_req_thread, NULL, mpidi_finalize_req, NULL); - /*MPIU_THREAD_CS_EXIT(ALLFUNC,); */ - while (mpidi_sync_done !=1) { - mpi_errno=PAMI_Context_advance(MPIDI_Context[0], 1000); - if (mpi_errno == PAMI_EAGAIN) { - usleep(1); - } - } - - if (mpi_errno = pthread_join(finalize_req_thread, NULL) ) { - TRACE_ERR("error returned from pthread_join() mpi_errno=%d\n",mpi_errno); - } -#endif - MPIU_THREAD_CS_EXIT(ALLFUNC,); - PMI2_Finalize(); - MPIU_THREAD_CS_ENTER(ALLFUNC,); - - if(_conn_info_list) { - if(_conn_info_list->rem_taskids) - MPL_free(_conn_info_list->rem_taskids); - else - MPL_free(_conn_info_list); - } - /* Free the storage associated with the process groups */ - pg = MPIDI_PG_list; - while (pg) { - pgNext = pg->next; - - /* In finalize, we free all process group information, even if - the ref count is not zero. This can happen if the user - fails to use MPI_Comm_disconnect on communicators that - were created with the dynamic process routines.*/ - /* XXX DJG FIXME-MT should we be checking this? */ - if (MPIR_Object_get_ref(pg) == 0 ) { - if (pg == MPIDI_Process.my_pg) - MPIDI_Process.my_pg = NULL; - MPIR_Object_set_ref(pg, 0); /* satisfy assertions in PG_Destroy */ - MPIDI_PG_Destroy( pg ); - } - pg = pgNext; - } - - /* If COMM_WORLD is still around (it normally should be), - try to free it here. The reason that we need to free it at this - point is that comm_world (and comm_self) still exist, and - hence the usual process to free the related VC structures will - not be invoked. */ - - /* The process group associated with MPI_COMM_WORLD will be - freed when MPI_COMM_WORLD is freed */ - - return mpi_errno; -} - - -/* This routine creates a new process group description and appends it to - the list of the known process groups. The pg_id is saved, not copied. - The PG_Destroy routine that was set with MPIDI_PG_Init is responsible for - freeing any storage associated with the pg_id. - - The new process group is returned in pg_ptr -*/ -int MPIDI_PG_Create(int vct_sz, void * pg_id, MPIDI_PG_t ** pg_ptr) -{ - MPIDI_PG_t * pg = NULL, *pgnext; - int p, i, j; - int mpi_errno = MPI_SUCCESS; - char *cp, *world_tasks, *cp1; - - pg = MPL_malloc(sizeof(MPIDI_PG_t)); - pg->vct = MPL_malloc(sizeof(struct MPID_VCR_t)*vct_sz); - - pg->handle = 0; - /* The reference count indicates the number of vc's that are or - have been in use and not disconnected. It starts at zero, - except for MPI_COMM_WORLD. */ - MPIR_Object_set_ref(pg, 0); - pg->size = vct_sz; - pg->id = MPL_strdup(pg_id); - TRACE_ERR("PG_Create - pg=%x pg->id=%s pg->vct=%x\n", pg, pg->id, pg->vct); - /* Initialize the connection information to null. Use - the appropriate MPIDI_PG_InitConnXXX routine to set up these - fields */ - - pg->connData = 0; - pg->getConnInfo = 0; - pg->connInfoToString = 0; - pg->connInfoFromString = 0; - pg->freeConnInfo = 0; - - for (p = 0; p < vct_sz; p++) - { - /* Initialize device fields in the VC object */ - MPIDI_VC_Init(&pg->vct[p], pg,p); - } - - /* The first process group is always the world group */ - if (!pg_world) { pg_world = pg; } - - /* Add pg's at the tail so that comm world is always the first pg */ - pg->next = 0; - if (!MPIDI_PG_list) - { - MPIDI_PG_list = pg; - } - else - { - pgnext = MPIDI_PG_list; - while (pgnext->next) - { - pgnext = pgnext->next; - } - pgnext->next = pg; - } - /* These are now done in MPIDI_VC_Init */ - *pg_ptr = pg; - - fn_exit: - return mpi_errno; - - fn_fail: - goto fn_exit; -} - -int MPIDI_PG_Destroy(MPIDI_PG_t * pg) -{ - MPIDI_PG_t * pg_prev; - MPIDI_PG_t * pg_cur; - int i; - int mpi_errno = MPI_SUCCESS; - - MPIR_Assert(MPIR_Object_get_ref(pg) == 0); - - pg_prev = NULL; - pg_cur = MPIDI_PG_list; - - while(pg_cur != NULL) - { - if (pg_cur == pg) - { - if (MPIDI_PG_iterator_next == pg) - { - MPIDI_PG_iterator_next = MPIDI_PG_iterator_next->next; - } - - if (pg_prev == NULL) - MPIDI_PG_list = pg->next; - else - pg_prev->next = pg->next; - - TRACE_ERR("destroying pg=%p pg->id=%s\n", pg, (char *)pg->id); - - for (i = 0; i < pg->size; ++i) { - /* FIXME it would be good if we could make this assertion. - Unfortunately, either: - 1) We're not being disciplined and some caller of this - function doesn't bother to manage all the refcounts - because he thinks he knows better. Annoying, but not - strictly a bug. - (wdg - actually, that is a bug - managing the ref - counts IS required and missing one is a bug.) - 2) There is a real bug lurking out there somewhere and we - just haven't hit it in the tests yet. */ - - /* This used to be handled in MPID_VCRT_Release, but that was - not the right place to do this. The VC should only be freed - when the PG that it belongs to is freed, not just when the - VC's refcount drops to zero. [goodell@ 2008-06-13] */ - /* In that case, the fact that the VC is in the PG should - increment the ref count - reflecting the fact that the - use in the PG constitutes a reference-count-incrementing - use. Alternately, if the PG is able to recreate a VC, - and can thus free unused (or idle) VCs, it should be allowed - to do so. [wdg 2008-08-31] */ - } - - MPIDI_PG_Destroy_fn(pg); - TRACE_ERR("destroying pg->vct=%x\n", pg->vct); - MPL_free(pg->vct); - TRACE_ERR("after destroying pg->vct=%x\n", pg->vct); - - if (pg->connData) { - if (pg->freeConnInfo) { - TRACE_ERR("calling freeConnInfo on pg\n"); - (*pg->freeConnInfo)( pg ); - } - else { - TRACE_ERR("free pg->connData\n"); - MPL_free(pg->connData); - } - } - - TRACE_ERR("final destroying pg\n"); - MPL_free(pg); - - goto fn_exit; - } - - pg_prev = pg_cur; - pg_cur = pg_cur->next; - } - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -int MPIDI_PG_Find(void * id, MPIDI_PG_t ** pg_ptr) -{ - MPIDI_PG_t * pg; - int mpi_errno = MPI_SUCCESS; - - pg = MPIDI_PG_list; - - while (pg != NULL) - { - if (MPIDI_PG_Compare_ids_fn(id, pg->id) != FALSE) - { - *pg_ptr = pg; - goto fn_exit; - } - - pg = pg->next; - } - - *pg_ptr = NULL; - - fn_exit: - return mpi_errno; -} - - -int MPIDI_PG_Id_compare(void * id1, void *id2) -{ - return MPIDI_PG_Compare_ids_fn(id1, id2); -} - -/* iter always points at the next element */ -int MPIDI_PG_Get_next(MPIDI_PG_iterator *iter, MPIDI_PG_t ** pg_ptr) -{ - *pg_ptr = (*iter); - if ((*iter) != NULL) { - (*iter) = (*iter)->next; - } - - return MPI_SUCCESS; -} - -int MPIDI_PG_Has_next(MPIDI_PG_iterator *iter) -{ - return (*iter != NULL); -} - -int MPIDI_PG_Get_iterator(MPIDI_PG_iterator *iter) -{ - *iter = MPIDI_PG_list; - return MPI_SUCCESS; -} - -/* FIXME: What does DEV_IMPLEMENTS_KVS mean? Why is it used? Who uses - PG_To_string and why? */ - -/* PG_To_string is used in the implementation of connect/accept (and - hence in spawn) */ -/* Note: Allocated memory that is returned in str_ptr. The user of - this routine must free that data */ -int MPIDI_PG_To_string(MPIDI_PG_t *pg_ptr, char **str_ptr, int *lenStr) -{ - int mpi_errno = MPI_SUCCESS; - - /* Replace this with the new string */ - MPIDI_connToStringKVS( str_ptr, lenStr, pg_ptr ); -#if 0 - if (pg_ptr->connInfoToString) { - (*pg_ptr->connInfoToString)( str_ptr, lenStr, pg_ptr ); - } - else { - MPIU_ERR_SETANDJUMP(mpi_errno,MPI_ERR_INTERN,"**noConnInfoToString"); - } -#endif - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -/* This routine takes a string description of a process group (created with - MPIDI_PG_To_string, usually on a different process) and returns a pointer to - the matching process group. If the group already exists, flag is set to - false. If the group does not exist, it is created with MPIDI_PG_Create (and - hence is added to the list of active process groups) and flag is set to - true. In addition, the connection information is set up using the - information in the input string. -*/ -int MPIDI_PG_Create_from_string(const char * str, MPIDI_PG_t ** pg_pptr, - int *flag) -{ - int mpi_errno = MPI_SUCCESS; - const char *p; - char *pg_id, *pg_id2, *cp2, *cp3,*str2, *str3; - int vct_sz, i; - MPIDI_PG_t *existing_pg, *pg_ptr=0; - - TRACE_ERR("MPIDI_PG_Create_from_string - str=%s\n", str); - /* The pg_id is at the beginning of the string, so we can just pass - it to the find routine */ - /* printf( "Looking for pg with id %s\n", str );fflush(stdout); */ - mpi_errno = MPIDI_PG_Find((void *)str, &existing_pg); - if (mpi_errno) TRACE_ERR("MPIDI_PG_Find returned with mpi_errno=%d\n", mpi_errno); - - if (existing_pg != NULL) { - /* return the existing PG */ - *pg_pptr = existing_pg; - *flag = 0; - /* Note that the memory for the pg_id is freed in the exit */ - goto fn_exit; - } - *flag = 1; - - /* Get the size from the string */ - p = str; - while (*p) p++; p++; - vct_sz = atoi(p); - - while (*p) p++;p++; - char *p_tmp = MPL_strdup(p); - TRACE_ERR("before MPIDI_PG_Create - p=%s p_tmp=%s vct_sz=%d\n", p, p_tmp, vct_sz); - mpi_errno = MPIDI_PG_Create(vct_sz, (void *)str, pg_pptr); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_PG_Create returned with mpi_errno=%d\n", mpi_errno); - } - - pg_ptr = *pg_pptr; - pg_ptr->vct[0].taskid=atoi(strtok(p_tmp,":")); - for(i=1; i<vct_sz; i++) { - pg_ptr->vct[i].taskid=atoi(strtok(NULL,":")); - } - TRACE_ERR("pg_ptr->id = %s\n",(*pg_pptr)->id); - MPL_free(p_tmp); - - if(verbose) - MPIU_PG_Printall(stderr); - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -#ifdef HAVE_CTYPE_H -/* Needed for isdigit */ -#include <ctype.h> -#endif - - -/* For all of these routines, the format of the process group description - that is created and used by the connTo/FromString routines is this: - (All items are strings, terminated by null) - - process group id string - sizeof process group (as string) - conninfo for rank 0 - conninfo for rank 1 - ... - - The "conninfo for rank 0" etc. for the original (MPI_COMM_WORLD) - process group are stored in the PMI_KVS space with the keys - p<rank>-businesscard . - - Fixme: Add a routine to publish the connection info to this file so that - the key for the businesscard is defined in just this one file. -*/ - - -/* The "KVS" versions are for the process group to which the calling - process belongs. These use the PMI_KVS routines to access the - process information */ -static int MPIDI_getConnInfoKVS( int rank, char *buf, int bufsize, MPIDI_PG_t *pg ) -{ -#ifdef USE_PMI2_API - char key[MPIDI_MAX_KVS_KEY_LEN]; - int mpi_errno = MPI_SUCCESS, rc; - int vallen; - - rc = MPL_snprintf(key, MPIDI_MAX_KVS_KEY_LEN, "P%d-businesscard", rank ); - - mpi_errno = PMI2_KVS_Get(pg->connData, PMI2_ID_NULL, key, buf, bufsize, &vallen); - if (mpi_errno) { - TRACE_ERR("PMI2_KVS_Get returned with mpi_errno=%d\n", mpi_errno); - } - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -#else - char key[MPIDI_MAX_KVS_KEY_LEN]; - int mpi_errno = MPI_SUCCESS, rc, pmi_errno; - - rc = MPL_snprintf(key, MPIDI_MAX_KVS_KEY_LEN, "P%d-businesscard", rank ); - if (rc < 0 || rc > MPIDI_MAX_KVS_KEY_LEN) { - MPIU_ERR_SETANDJUMP(mpi_errno,MPI_ERR_OTHER,"**nomem"); - } - -/* MPIU_THREAD_CS_ENTER(PMI,);*/ - pmi_errno = PMI_KVS_Get(pg->connData, key, buf, bufsize ); - if (pmi_errno) { - MPIDI_PG_CheckForSingleton(); - pmi_errno = PMI_KVS_Get(pg->connData, key, buf, bufsize ); - } -/* MPIU_THREAD_CS_EXIT(PMI,);*/ - if (pmi_errno) { - MPIU_ERR_SETANDJUMP(mpi_errno,MPI_ERR_OTHER,"**pmi_kvs_get"); - } - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -#endif -} - -/* *slen is the length of the string, including the null terminator. So if the - resulting string is |foo\0bar\0|, then *slen == 8. */ -int MPIDI_connToStringKVS( char **buf_p, int *slen, MPIDI_PG_t *pg ) -{ - char *string = 0; - char *pg_idStr = (char *)pg->id; /* In the PMI/KVS space, - the pg id is a string */ - char buf[MPIDI_MAX_KVS_VALUE_LEN]; - int i, j, vallen, rc, mpi_errno = MPI_SUCCESS, len; - int curSlen, nChars; - - /* Make an initial allocation of a string with an estimate of the - needed space */ - len = 0; - curSlen = 10 + pg->size * 128; - string = (char *)MPL_malloc( curSlen ); - - /* Start with the id of the pg */ - while (*pg_idStr && len < curSlen) - string[len++] = *pg_idStr++; - string[len++] = 0; - - /* Add the size of the pg */ - MPL_snprintf( &string[len], curSlen - len, "%d", pg->size ); - while (string[len]) len++; - string[len++] = 0; - - /* add the taskids of the pg */ - for(i = 0; i < pg->size; i++) { - MPL_snprintf(buf, MPIDI_MAX_KVS_VALUE_LEN, "%d:", pg->vct[i].taskid); - vallen = strlen(buf); - if (len+vallen+1 >= curSlen) { - char *nstring = 0; - curSlen += (pg->size - i) * (vallen + 1 ); - nstring = MPL_realloc( string, curSlen); - MPID_assert(nstring != NULL); - string = nstring; - } - /* Append to string */ - nChars = MPL_snprintf(&string[len], curSlen - len, "%d:", pg->vct[i].taskid); - len+=nChars; - } - -#if 0 - for (i=0; i<pg->size; i++) { - rc = getConnInfoKVS( i, buf, MPIDI_MAX_KVS_VALUE_LEN, pg ); - if (rc) { - MPL_internal_error_printf( - "Panic: getConnInfoKVS failed for %s (rc=%d)\n", - (char *)pg->id, rc ); - } -#ifndef USE_PERSISTENT_SHARED_MEMORY - /* FIXME: This is a hack to avoid including shared-memory - queue names in the business card that may be used - by processes that were not part of the same COMM_WORLD. - To fix this, the shared memory channels should look at the - returned connection info and decide whether to use - sockets or shared memory by determining whether the - process is in the same MPI_COMM_WORLD. */ - /* FIXME: The more general problem is that the connection information - needs to include some information on the range of validity (e.g., - all processes, same comm world, particular ranks), and that - representation needs to be scalable */ -/* printf( "Adding key %s value %s\n", key, val ); */ - { - char *p = strstr( buf, "$shm_host" ); - if (p) p[1] = 0; - /* printf( "(fixed) Adding key %s value %s\n", key, val ); */ - } -#endif - /* Add the information to the output buffer */ - vallen = strlen(buf); - /* Check that this will fix in the remaining space */ - if (len + vallen + 1 >= curSlen) { - char *nstring = 0; - curSlen += (pg->size - i) * (vallen + 1 ); - nstring = MPL_realloc( string, curSlen); - if (!nstring) { - MPIU_ERR_SETANDJUMP(mpi_errno,MPI_ERR_OTHER,"**nomem"); - } - string = nstring; - } - /* Append to string */ - for (j=0; j<vallen+1; j++) { - string[len++] = buf[j]; - } - } -#endif - - MPIR_Assert(len <= curSlen); - - *buf_p = string; - *slen = len; - fn_exit: - return mpi_errno; - fn_fail: - if (string) MPL_free(string); - goto fn_exit; -} - -static int MPIDI_connFromStringKVS( const char *buf ATTRIBUTE((unused)), - MPIDI_PG_t *pg ATTRIBUTE((unused)) ) -{ - /* Fixme: this should be a failure to call this routine */ - return MPI_SUCCESS; -} -static int MPIDI_connFreeKVS( MPIDI_PG_t *pg ) -{ - if (pg->connData) { - MPL_free( pg->connData ); - } - return MPI_SUCCESS; -} - - -int MPIDI_PG_InitConnKVS( MPIDI_PG_t *pg ) -{ -#ifdef USE_PMI2_API - int mpi_errno = MPI_SUCCESS; - - pg->connData = (char *)MPL_malloc(MAX_JOBID_LEN); - if (pg->connData == NULL) { - TRACE_ERR("MPIDI_PG_InitConnKVS - MPL_malloc failure\n"); - } - - mpi_errno = PMI2_Job_GetId(pg->connData, MAX_JOBID_LEN); - if (mpi_errno) TRACE_ERR("PMI2_Job_GetId returned with mpi_errno=%d\n", mpi_errno); -#else - int pmi_errno, kvs_name_sz; - int mpi_errno = MPI_SUCCESS; - - pmi_errno = PMI_KVS_Get_name_length_max( &kvs_name_sz ); - if (pmi_errno != PMI_SUCCESS) { - MPIU_ERR_SETANDJUMP1(mpi_errno,MPI_ERR_OTHER, - "**pmi_kvs_get_name_length_max", - "**pmi_kvs_get_name_length_max %d", pmi_errno); - } - - pg->connData = (char *)MPL_malloc(kvs_name_sz + 1); - if (pg->connData == NULL) { - MPIU_ERR_SETANDJUMP(mpi_errno,MPI_ERR_OTHER, "**nomem"); - } - - pmi_errno = PMI_KVS_Get_my_name(pg->connData, kvs_name_sz); - if (pmi_errno != PMI_SUCCESS) { - MPIU_ERR_SETANDJUMP1(mpi_errno,MPI_ERR_OTHER, - "**pmi_kvs_get_my_name", - "**pmi_kvs_get_my_name %d", pmi_errno); - } -#endif - pg->getConnInfo = MPIDI_getConnInfoKVS; - pg->connInfoToString = MPIDI_connToStringKVS; - pg->connInfoFromString = MPIDI_connFromStringKVS; - pg->freeConnInfo = MPIDI_connFreeKVS; - - fn_exit: - return mpi_errno; - fn_fail: - if (pg->connData) { MPL_free(pg->connData); } - goto fn_exit; -} - -/* Return the kvsname associated with the MPI_COMM_WORLD of this process. */ -int MPIDI_PG_GetConnKVSname( char ** kvsname ) -{ - *kvsname = pg_world->connData; - return MPI_SUCCESS; -} - -/* For process groups that are not our MPI_COMM_WORLD, store the connection - information in an array of strings. These routines and structure - implement the access to this information. */ -typedef struct { - int toStringLen; /* Length needed to encode this connection info */ - char ** connStrings; /* pointer to an array, indexed by rank, containing - connection information */ -} MPIDI_ConnInfo; - - -static int MPIDI_getConnInfo( int rank, char *buf, int bufsize, MPIDI_PG_t *pg ) -{ - MPIDI_ConnInfo *connInfo = (MPIDI_ConnInfo *)pg->connData; - - /* printf( "Entering getConnInfo\n" ); fflush(stdout); */ - if (!connInfo || !connInfo->connStrings || !connInfo->connStrings[rank]) { - /* FIXME: Turn this into a valid error code create/return */ - /*printf( "Fatal error in getConnInfo (rank = %d)\n", rank ); - printf( "connInfo = %p\n", connInfo );fflush(stdout); */ - if (connInfo) { -/* printf( "connInfo->connStrings = %p\n", connInfo->connStrings ); */ - } - /* Fatal error. Connection information missing */ - fflush(stdout); - } - - /* printf( "Copying %s to buf\n", connInfo->connStrings[rank] ); fflush(stdout); */ - - MPL_strncpy( buf, connInfo->connStrings[rank], bufsize ); - return MPI_SUCCESS; -} - - -static int MPIDI_connToString( char **buf_p, int *slen, MPIDI_PG_t *pg ) -{ - int mpi_errno = MPI_SUCCESS; - char *str = NULL, *pg_id; - int i, len=0; - MPIDI_ConnInfo *connInfo = (MPIDI_ConnInfo *)pg->connData; - - /* Create this from the string array */ - str = (char *)MPL_malloc(connInfo->toStringLen); - -#if defined(MPICH_DEBUG_MEMINIT) - memset(str, 0, connInfo->toStringLen); -#endif - - pg_id = pg->id; - /* FIXME: This is a hack, and it doesn't even work */ - /* MPIDI_PrintConnStrToFile( stdout, __FILE__, __LINE__, - "connToString: pg id is", (char *)pg_id );*/ - /* This is intended to cause a process to transition from a singleton - to a non-singleton. */ - /* XXX DJG TODO figure out what this little bit is all about. */ - if (strstr( pg_id, "singinit_kvs" ) == pg_id) { -#ifdef USE_PMI2_API - MPIR_Assertp(0); /* don't know what to do here for pmi2 yet. DARIUS */ -#else - PMI_KVS_Get_my_name( pg->id, 256 ); -#endif - } - - while (*pg_id) str[len++] = *pg_id++; - str[len++] = 0; - - MPL_snprintf( &str[len], 20, "%d", pg->size); - /* Skip over the length */ - while (str[len++]); - - /* Copy each connection string */ - for (i=0; i<pg->size; i++) { - char *p = connInfo->connStrings[i]; - while (*p) { str[len++] = *p++; } - str[len++] = 0; - } - - if (len > connInfo->toStringLen) { - *buf_p = 0; - *slen = 0; - TRACE_ERR("len > connInfo->toStringLen"); - } - - *buf_p = str; - *slen = len; - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; - -} - - -static int MPIDI_connFromString( const char *buf, MPIDI_PG_t *pg ) -{ - MPIDI_ConnInfo *conninfo = 0; - int i, mpi_errno = MPI_SUCCESS; - const char *buf0 = buf; /* save the start of buf */ - - /* printf( "Starting with buf = %s\n", buf );fflush(stdout); */ - - /* Skip the pg id */ - while (*buf) buf++; buf++; - - /* Determine the size of the pg */ - pg->size = atoi( buf ); - while (*buf) buf++; buf++; - - conninfo = (MPIDI_ConnInfo *)MPL_malloc( sizeof(MPIDI_ConnInfo) ); - conninfo->connStrings = (char **)MPL_malloc( pg->size * sizeof(char *)); - - /* For now, make a copy of each item */ - for (i=0; i<pg->size; i++) { - /* printf( "Adding conn[%d] = %s\n", i, buf );fflush(stdout); */ - conninfo->connStrings[i] = MPL_strdup( buf ); - while (*buf) buf++; - buf++; - } - pg->connData = conninfo; - - /* Save the length of the string needed to encode the connection - information */ - conninfo->toStringLen = (int)(buf - buf0) + 1; - - return mpi_errno; -} - - -static int MPIDI_connFree( MPIDI_PG_t *pg ) -{ - MPIDI_ConnInfo *conninfo = (MPIDI_ConnInfo *)pg->connData; - int i; - - for (i=0; i<pg->size; i++) { - MPL_free( conninfo->connStrings[i] ); - } - MPL_free( conninfo->connStrings ); - MPL_free( conninfo ); - - return MPI_SUCCESS; -} - - -/*@ - MPIDI_PG_Dup_vcr - Duplicate a virtual connection from a process group - - Notes: - This routine provides a dup of a virtual connection given a process group - and a rank in that group. This routine is used only in initializing - the MPI-1 communicators 'MPI_COMM_WORLD' and 'MPI_COMM_SELF', and in creating - the initial intercommunicator after an 'MPI_Comm_spawn', - 'MPI_Comm_spawn_multiple', or 'MPI_Comm_connect/MPI_Comm_accept'. - - In addition to returning a dup of the virtual connection, it manages the - reference count of the process group, which is always the number of inuse - virtual connections. - @*/ -int MPIDI_PG_Dup_vcr( MPIDI_PG_t *pg, int rank, pami_task_t taskid, MPID_VCR *vcr_p ) -{ - int inuse; - MPID_VCR vcr; - - TRACE_ERR("ENTER MPIDI_PG_Dup_vcr - pg->id=%s rank=%d taskid=%d\n", pg->id, rank, taskid); - pg->vct[rank].taskid = taskid; - - vcr = MPL_malloc(sizeof(struct MPID_VCR_t)); - TRACE_ERR("MPIDI_PG_Dup_vcr- pg->vct[%d].pg=%x pg=%x vcr=%x vcr->pg=%x\n", rank, pg->vct[rank].pg, pg, vcr, vcr->pg); - vcr->pg = pg; - vcr->pg_rank = rank; - vcr->taskid = taskid; - /* Increase the reference count of the vc. If the reference count - increases from 0 to 1, increase the reference count of the - process group *and* the reference count of the vc (this - allows us to distinquish between Comm_free and Comm_disconnect) */ - /* FIXME-MT: This should be a fetch and increment for thread-safety */ - /*if (MPIR_Object_get_ref(vcr_p) == 0) { */ - TRACE_ERR("MPIDI_PG_add_ref on pg=%s pg=%x\n", pg->id, pg); - MPIDI_PG_add_ref(pg); - inuse=MPIR_Object_get_ref(pg); - TRACE_ERR("after MPIDI_PG_add_ref on pg=%s inuse=%d\n", pg->id, inuse); -/* MPIDI_VC_add_ref(vcr_p); - } - MPIDI_VC_add_ref(vcr_p);*/ - *vcr_p = vcr; - - return MPI_SUCCESS; -} - - -/* - * This routine may be called to print the contents (including states and - * reference counts) for process groups. - */ -int MPIU_PG_Printall( FILE *fp ) -{ - MPIDI_PG_t *pg; - int i; - - pg = MPIDI_PG_list; - - fprintf( fp, "Process groups:\n" ); - while (pg) { - /* XXX DJG FIXME-MT should we be checking this? */ - fprintf( fp, "size = %d, refcount = %d, id = %s\n", - pg->size, MPIR_Object_get_ref(pg), (char *)pg->id ); - for (i=0; i<pg->size; i++) { - fprintf( fp, "\tVCT rank = %d, refcount = %d, taskid = %d\n", - pg->vct[i].pg_rank, MPIR_Object_get_ref(pg), - pg->vct[i].taskid ); - } - fflush(fp); - pg = pg->next; - } - - return 0; -} - -#ifdef HAVE_CTYPE_H -/* Needed for isdigit */ -#include <ctype.h> -#endif -/* Convert a process group id into a number. This is a hash-based approach, - * which has the potential for some collisions. This is an alternative to the - * previous approach that caused req#3930, which was to sum up the values of the - * characters. The summing approach worked OK when the id's were all similar - * but with an incrementing prefix or suffix, but terrible for a 32 hex-character - * UUID type of id. - * - * FIXME It would really be best if the PM could give us this value. - */ -/* FIXME: This is a temporary hack for devices that do not define - MPIDI_DEV_IMPLEMENTS_KVS - FIXME: MPIDI_DEV_IMPLEMENTS_KVS should be removed - */ -void MPIDI_PG_IdToNum( MPIDI_PG_t *pg, int *id ) -{ - *id = atoi((char *)pg->id); -} - - -int MPID_PG_ForwardPGInfo( MPIR_Comm *peer_ptr, MPIR_Comm *comm_ptr, - int nPGids, const int gpids[], - int root ) -{ - int mpi_errno = MPI_SUCCESS; - int i, allfound = 1, pgid, pgidWorld; - MPIDI_PG_t *pg = 0; - MPIDI_PG_iterator iter; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - - if(mpidi_dynamic_tasking) { - /* Get the pgid for CommWorld (always attached to the first process - group) */ - MPIDI_PG_Get_iterator(&iter); - MPIDI_PG_Get_next( &iter, &pg ); - MPIDI_PG_IdToNum( pg, &pgidWorld ); - - /* Extract the unique process groups */ - for (i=0; i<nPGids && allfound; i++) { - if (gpids[0] != pgidWorld) { - /* Add this gpid to the list of values to check */ - /* FIXME: For testing, we just test in place */ - MPIDI_PG_Get_iterator(&iter); - do { - MPIDI_PG_Get_next( &iter, &pg ); - if (!pg) { - /* We don't know this pgid */ - allfound = 0; - break; - } - MPIDI_PG_IdToNum( pg, &pgid ); - } while (pgid != gpids[0]); - } - gpids += 2; - } - - /* See if everyone is happy */ - mpi_errno = MPIR_Allreduce_impl( MPI_IN_PLACE, &allfound, 1, MPI_INT, MPI_LAND, comm_ptr, &errflag ); - - if (allfound) return MPI_SUCCESS; - - /* FIXME: We need a cleaner way to handle this case than using an ifdef. - We could have an empty version of MPID_PG_BCast in ch3u_port.c, but - that's a rather crude way of addressing this problem. Better is to - make the handling of local and remote PIDS for the dynamic process - case part of the dynamic process "module"; devices that don't support - dynamic processes (and hence have only COMM_WORLD) could optimize for - that case */ - /* We need to share the process groups. We use routines - from ch3u_port.c */ - MPID_PG_BCast( peer_ptr, comm_ptr, root ); - } - fn_exit: - return MPI_SUCCESS; - fn_fail: - goto fn_exit; -} -#endif diff --git a/src/mpid/pamid/src/dyntask/mpidi_port.c b/src/mpid/pamid/src/dyntask/mpidi_port.c deleted file mode 100644 index d539323..0000000 --- a/src/mpid/pamid/src/dyntask/mpidi_port.c +++ /dev/null @@ -1,1766 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ - -#include <mpidimpl.h> - -#ifdef DYNAMIC_TASKING -#define MAX_HOST_DESCRIPTION_LEN 256 -#define WORLDINTCOMMCNTR _global_world_intercomm_cntr -#ifdef USE_PMI2_API -#define MPID_MAX_JOBID_LEN 256 -#define TOTAL_AM 3 -#endif - -transactionID_struct *_transactionID_list = NULL; - -typedef struct { - MPID_VCR vcr; - int port_name_tag; -}AM_struct; - -conn_info *_conn_info_list = NULL; -extern int mpidi_dynamic_tasking; -long long _global_world_intercomm_cntr = 0; - -typedef struct MPIDI_Acceptq -{ - int port_name_tag; - MPID_VCR vcr; - struct MPIDI_Acceptq *next; -} -MPIDI_Acceptq_t; - -static MPIDI_Acceptq_t * acceptq_head=0; -static int maxAcceptQueueSize = 0; -static int AcceptQueueSize = 0; - -pthread_mutex_t rem_connlist_mutex = PTHREAD_MUTEX_INITIALIZER; - -/* FIXME: If dynamic processes are not supported, this file will contain - no code and some compilers may warn about an "empty translation unit" */ - -/* FIXME: pg_translation is used for ? */ -typedef struct pg_translation { - int pg_index; /* index of a process group (index in pg_node) */ - int pg_rank; /* rank in that process group */ - pami_task_t pg_taskid; /* rank in that process group */ -} pg_translation; - - -typedef struct pg_node { - int index; /* Internal index of process group - (see pg_translation) */ - char *pg_id; - char *str; /* String describing connection info for pg */ - int lenStr; /* Length of this string (including the null terminator(s)) */ - struct pg_node *next; -} pg_node; - - -void MPIDI_Recvfrom_remote_world(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - AM_struct *AM_data; - MPID_VCR *new_vcr; - int port_name_tag; - MPIDI_Acceptq_t *q_item; - pami_endpoint_t dest; - - - q_item = MPL_malloc(sizeof(MPIDI_Acceptq_t)); - q_item->vcr = MPL_malloc(sizeof(struct MPID_VCR_t)); - q_item->vcr->pg = MPL_malloc(sizeof(MPIDI_PG_t)); - MPIR_Object_set_ref(q_item->vcr->pg, 0); - TRACE_ERR("ENTER MPIDI_Acceptq_enqueue-1 q_item=%llx _msginfo=%llx (AM_struct *)_msginfo=%llx ((AM_struct *)_msginfo)->vcr=%llx\n", q_item, _msginfo, (AM_struct *)_msginfo, ((AM_struct *)_msginfo)->vcr); - q_item->port_name_tag = ((AM_struct *)_msginfo)->port_name_tag; - q_item->vcr->taskid = PAMIX_Endpoint_query(sender); - TRACE_ERR("MPIDI_Recvfrom_remote_world INVOKED with new_vcr->taskid=%d\n",sender); - - /* Keep some statistics on the accept queue */ - AcceptQueueSize++; - if (AcceptQueueSize > maxAcceptQueueSize) - maxAcceptQueueSize = AcceptQueueSize; - - q_item->next = acceptq_head; - acceptq_head = q_item; - return; -} - - -/* These functions help implement the connect/accept algorithm */ -static int MPIDI_ExtractLocalPGInfo( struct MPIR_Comm *, pg_translation [], - pg_node **, int * ); -static int MPIDI_ReceivePGAndDistribute( struct MPIR_Comm *, struct MPIR_Comm *, int, int *, - int, MPIDI_PG_t *[] ); -static int MPIDI_SendPGtoPeerAndFree( struct MPIR_Comm *, int *, pg_node * ); -static int MPIDI_SetupNewIntercomm( struct MPIR_Comm *comm_ptr, int remote_comm_size, - pg_translation remote_translation[], - int n_remote_pgs, MPIDI_PG_t **remote_pg, - struct MPIR_Comm *intercomm ); -static int MPIDI_Initialize_tmp_comm(struct MPIR_Comm **comm_pptr, - struct MPID_VCR_t *vcr_ptr, int is_low_group, int context_id_offset); - - -/* ------------------------------------------------------------------------- */ -/* - * Structure of this file and the connect/accept algorithm: - * - * Here are the steps involved in implementating MPI_Comm_connect and - * MPI_Comm_accept. These same steps are used withing MPI_Comm_spawn - * and MPI_Comm_spawn_multiple. - * - * First, the connecting process establishes a connection (not a virtual - * connection!) to the designated accepting process. - * This makes use of the usual (channel-specific) connection code. - * Once this connection is established, the connecting process sends a packet - * to the accepting process. - * This packet contains a "port_tag_name", which is a value that - * is used to separate different MPI port names (values from MPI_Open_port) - * on the same process (this is a way to multiplex many MPI port names on - * a single communication connection port). - * - * On the accepting side, the process waits until the progress engine - * inserts the connect request into the accept queue (this is done with the - * routine MPIDI_Acceptq_dequeue). This routine returns the matched - * virtual connection (VC). - * - * Once both sides have established there VC, they both invoke - * MPIDI_Initialize_tmp_comm to create a temporary intercommunicator. - * A temporary intercommunicator is constructed so that we can use - * MPI routines to send the other information that we need to complete - * the connect/accept operation (described below). - * - * The above is implemented with the routines - * MPIDI_Create_inter_root_communicator_connect - * MPIDI_Create_inter_root_communicator_accept - * MPIDI_Initialize_tmp_comm - * - * At this point, the two "root" processes of the communicators that are - * connecting can use MPI communication. They must then exchange the - * following information: - * - * The size of the "remote" communicator - * Description of all process groups; that is, all of the MPI_COMM_WORLDs - * that they know. - * The shared context id that will be used - * - * - */ -/* ------------------------------------------------------------------------- */ - -int MPIDU_send_AM_to_leader(MPID_VCR new_vcr, int port_name_tag, pami_task_t taskid) -{ - pami_send_t xferP; - pami_endpoint_t dest; - int rc, current_val; - - AM_struct AM_data; - - AM_data.vcr = new_vcr; - TRACE_ERR("MPIDU_send_AM_to_leader - new_vcr->taskid=%d\n", new_vcr->taskid); - AM_data.port_name_tag = port_name_tag; - TRACE_ERR("send - %p %d %p %d\n", AM_data.vcr, AM_data.port_name_tag, AM_data.vcr, AM_data.vcr->taskid); - - - memset(&xferP, 0, sizeof(pami_send_t)); - xferP.send.header.iov_base = (void*)&AM_data; - xferP.send.header.iov_len = sizeof(AM_struct); - xferP.send.dispatch = MPIDI_Protocols_Dyntask; - /*xferP.hints.use_rdma = mpci_enviro.use_shmem; - xferP.hints.use_shmem = mpci_enviro.use_shmem;*/ - rc = PAMI_Endpoint_create(MPIDI_Client, taskid, 0, &dest); - TRACE_ERR("PAMI_Resume to taskid=%d\n", taskid); - PAMI_Resume(MPIDI_Context[0], - &dest, 1); - - if(rc != 0) - TRACE_ERR("PAMI_Endpoint_create failed\n"); - - xferP.send.dest = dest; - - rc = PAMI_Send(MPIDI_Context[0], &xferP); - -} - - -/* - * These next two routines are used to create a virtual connection - * (VC) and a temporary intercommunicator that can be used to - * communicate between the two "root" processes for the - * connect and accept. - */ - -/* FIXME: Describe the algorithm for the connection logic */ -int MPIDI_Connect_to_root(const char * port_name, - MPID_VCR * new_vc) -{ - int mpi_errno = MPI_SUCCESS; - MPID_VCR vc; - char host_description[MAX_HOST_DESCRIPTION_LEN]; - int port, port_name_tag; pami_task_t taskid_tag; - int hasIfaddr = 0; - AM_struct *conn; - - /* First, create a new vc (we may use this to pass to a generic - connection routine) */ - vc = MPL_malloc(sizeof(struct MPID_VCR_t)); - vc->pg = MPL_malloc(sizeof(MPIDI_PG_t)); - MPIR_Object_set_ref(vc->pg, 0); - TRACE_ERR("vc from MPIDI_Connect_to_root=%llx vc->pg=%llx\n", vc, vc->pg); - /* FIXME - where does this vc get freed? */ - - *new_vc = vc; - - /* FIXME: There may need to be an additional routine here, to ensure that the - channel is initialized for this pair of process groups (this process - and the remote process to which the vc will connect). */ -/* MPIDI_VC_Init(vc->vc, NULL, 0); */ - vc->taskid = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval; - TRACE_ERR("MPIDI_Connect_to_root - vc->taskid=%d\n", vc->taskid); - - mpi_errno = MPIDI_GetTagFromPort(port_name, &port_name_tag); - if (mpi_errno != MPL_STR_SUCCESS) { - TRACE_ERR("MPIDI_GetTagFromPort returned with mpi_errno=%d", mpi_errno); - } - mpi_errno = MPIDI_GetTaskidFromPort(port_name, &taskid_tag); - if (mpi_errno != MPL_STR_SUCCESS) { - TRACE_ERR("MPIDI_GetTaskidFromPort returned with mpi_errno=%d", mpi_errno); - } - - TRACE_ERR("posting connect to host %s, port %d task %d vc %p\n", - host_description, port, taskid_tag, vc ); - mpi_errno = MPIDU_send_AM_to_leader(vc, port_name_tag, taskid_tag); - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -/* ------------------------------------------------------------------------- */ -/* Business card management. These routines insert or extract connection - information when using sockets from the business card */ -/* ------------------------------------------------------------------------- */ - -/* FIXME: These are small routines; we may want to bring them together - into a more specific post-connection-for-sock */ - -/* The host_description should be of length MAX_HOST_DESCRIPTION_LEN */ - - -static int MPIDI_Create_inter_root_communicator_connect(const char *port_name, - struct MPIR_Comm **comm_pptr, - MPID_VCR *vc_pptr) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIR_Comm *tmp_comm; - struct MPID_VCR_t *connect_vc= NULL; - int port_name_tag, taskid_tag; - /* Connect to the root on the other side. Create a - temporary intercommunicator between the two roots so that - we can use MPI functions to communicate data between them. */ - - MPIDI_Connect_to_root(port_name, &(connect_vc)); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_Connect_to_root returned with mpi_errno=%d\n", mpi_errno); - } - - /* extract the tag from the port_name */ - mpi_errno = MPIDI_GetTagFromPort( port_name, &port_name_tag); - if (mpi_errno != MPL_STR_SUCCESS) { - TRACE_ERR("MPIDI_GetTagFromPort returned with mpi_errno=%d\n", mpi_errno); - } - - mpi_errno = MPIDI_GetTaskidFromPort(port_name, &taskid_tag); - if (mpi_errno != MPL_STR_SUCCESS) { - TRACE_ERR("MPIDI_GetTaskidFromPort returned with mpi_errno=%d\n", mpi_errno); - } - connect_vc->taskid=taskid_tag; - mpi_errno = MPIDI_Initialize_tmp_comm(&tmp_comm, connect_vc, 1, port_name_tag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_Initialize_tmp_comm returned with mpi_errno=%d\n", mpi_errno); - } - - *comm_pptr = tmp_comm; - *vc_pptr = connect_vc; - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -/* Creates a communicator for the purpose of communicating with one other - process (the root of the other group). It also returns the virtual - connection */ -static int MPIDI_Create_inter_root_communicator_accept(const char *port_name, - struct MPIR_Comm **comm_pptr, - MPID_VCR *vc_pptr) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIR_Comm *tmp_comm; - MPID_VCR new_vc; - - MPID_Progress_state progress_state; - int port_name_tag; - - /* extract the tag from the port_name */ - mpi_errno = MPIDI_GetTagFromPort( port_name, &port_name_tag); - if (mpi_errno != MPL_STR_SUCCESS) { - TRACE_ERR("MPIDI_GetTagFromPort returned with mpi_errnp=%d\n", mpi_errno); - } - - /* FIXME: Describe the algorithm used here, and what routine - is user on the other side of this connection */ - /* dequeue the accept queue to see if a connection with the - root on the connect side has been formed in the progress - engine (the connection is returned in the form of a vc). If - not, poke the progress engine. */ - - for(;;) - { - MPIDI_Acceptq_dequeue(&new_vc, port_name_tag); - if (new_vc != NULL) - { - break; - } - - mpi_errno = MPID_Progress_wait(100); - /* --BEGIN ERROR HANDLING-- */ - if (mpi_errno) - { - TRACE_ERR("MPID_Progress_wait returned with mpi_errno=%d\n", mpi_errno); - } - /* --END ERROR HANDLING-- */ - } - - mpi_errno = MPIDI_Initialize_tmp_comm(&tmp_comm, new_vc, 0, port_name_tag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_Initialize_tmp_comm returned with mpi_errno=%d\n", mpi_errno); - } - - *comm_pptr = tmp_comm; - *vc_pptr = new_vc; - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -/* This is a utility routine used to initialize temporary communicators - used in connect/accept operations, and is only used in the above two - routines */ -static int MPIDI_Initialize_tmp_comm(struct MPIR_Comm **comm_pptr, - struct MPID_VCR_t *vc_ptr, int is_low_group, int context_id_offset) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIR_Comm *tmp_comm, *commself_ptr; - - MPIR_Comm_get_ptr( MPI_COMM_SELF, commself_ptr ); - - /* WDG-old code allocated a context id that was then discarded */ - mpi_errno = MPIR_Comm_create(&tmp_comm); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Comm_create returned with mpi_errno=%d\n", mpi_errno); - } - /* fill in all the fields of tmp_comm. */ - - /* We use the second half of the context ID bits for dynamic - * processes. This assumes that the context ID mask array is made - * up of uint32_t's. */ - /* FIXME: This code is still broken for the following case: - * If the same process opens connections to the multiple - * processes, this context ID might get out of sync. - */ - tmp_comm->context_id = MPIR_CONTEXT_SET_FIELD(DYNAMIC_PROC, context_id_offset, 1); - tmp_comm->recvcontext_id = tmp_comm->context_id; - - /* sanity: the INVALID context ID value could potentially conflict with the - * dynamic proccess space */ - MPIR_Assert(tmp_comm->context_id != MPIR_INVALID_CONTEXT_ID); - MPIR_Assert(tmp_comm->recvcontext_id != MPIR_INVALID_CONTEXT_ID); - - /* FIXME - we probably need a unique context_id. */ - tmp_comm->remote_size = 1; - - /* Fill in new intercomm */ - tmp_comm->local_size = 1; - tmp_comm->rank = 0; - tmp_comm->comm_kind = MPIR_COMM_KIND__INTERCOMM; - tmp_comm->local_comm = NULL; - tmp_comm->is_low_group = is_low_group; - - /* No pg structure needed since vc has already been set up - (connection has been established). */ - - /* Point local vcr, vcrt at those of commself_ptr */ - /* FIXME: Explain why */ - tmp_comm->local_vcrt = commself_ptr->vcrt; - MPID_VCRT_Add_ref(commself_ptr->vcrt); - tmp_comm->local_vcr = commself_ptr->vcr; - - /* No pg needed since connection has already been formed. - FIXME - ensure that the comm_release code does not try to - free an unallocated pg */ - - /* Set up VC reference table */ - mpi_errno = MPID_VCRT_Create(tmp_comm->remote_size, &tmp_comm->vcrt); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPID_VCRT_Create returned with mpi_errno=%d", mpi_errno); - } - mpi_errno = MPID_VCRT_Get_ptr(tmp_comm->vcrt, &tmp_comm->vcr); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPID_VCRT_Get_ptr returned with mpi_errno=%d", mpi_errno); - } - - /* FIXME: Why do we do a dup here? */ - MPID_VCR_Dup(vc_ptr, tmp_comm->vcr); - - *comm_pptr = tmp_comm; - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - - -/** - * * Function to add a new trasaction id in the transaction id list. This function - * * gets called only when a new connection is made with remote tasks. - * */ -void MPIDI_add_new_tranid(long long tranid) -{ - int i; - transactionID_struct *tridtmp=NULL; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - if(_transactionID_list == NULL) { - _transactionID_list = (transactionID_struct*) MPL_malloc(sizeof(transactionID_struct)); - _transactionID_list->cntr_for_AM = MPL_malloc(TOTAL_AM*sizeof(int)); - _transactionID_list->tranid = tranid; - for(i=0;i<TOTAL_AM;i++) - _transactionID_list->cntr_for_AM[i] = 0; - _transactionID_list->next = NULL; - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - return; - } - - tridtmp = _transactionID_list; - while(tridtmp->next != NULL) - tridtmp = tridtmp->next; - - tridtmp->next = (transactionID_struct*) MPL_malloc(sizeof(transactionID_struct)); - tridtmp = tridtmp->next; - tridtmp->tranid = tranid; - tridtmp->cntr_for_AM = MPL_malloc(TOTAL_AM*sizeof(int)); - for(i=0;i<TOTAL_AM;i++) - tridtmp->cntr_for_AM[i] = 0; - tridtmp->next = NULL; - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); -} - - -/* ------------------------------------------------------------------------- */ -/* - MPIDI_Comm_connect() - - Algorithm: First create a connection (vc) between this root and the - root on the accept side. Using this vc, create a temporary - intercomm between the two roots. Use MPI functions to communicate - the other information needed to create the real intercommunicator - between the processes on the two sides. Then free the - intercommunicator between the roots. Most of the complexity is - because there can be multiple process groups on each side. -*/ -int MPIDI_Comm_connect(const char *port_name, MPIR_Info *info, int root, - struct MPIR_Comm *comm_ptr, struct MPIR_Comm **newcomm) -{ - int mpi_errno=MPI_SUCCESS; - int j, i, rank, recv_ints[3], send_ints[3], context_id; - int remote_comm_size=0; - struct MPIR_Comm *tmp_comm = NULL; - MPID_VCR new_vc= NULL; - int sendtag=100, recvtag=100, n_remote_pgs; - int n_local_pgs=1, local_comm_size; - pg_translation *local_translation = NULL, *remote_translation = NULL; - pg_node *pg_list = NULL; - MPIDI_PG_t **remote_pg = NULL; - MPIR_Context_id_t recvcontext_id = MPIR_INVALID_CONTEXT_ID; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - long long comm_cntr, lcomm_cntr; - - /* Get the context ID here because we need to send it to the remote side */ - mpi_errno = MPIR_Get_contextid( comm_ptr, &recvcontext_id ); - TRACE_ERR("MPIDI_Comm_connect calling MPIR_Get_contextid = %d\n", recvcontext_id); - if (mpi_errno) TRACE_ERR("MPIR_Get_contextid returned with mpi_errno=%d\n", mpi_errno); - - rank = comm_ptr->rank; - local_comm_size = comm_ptr->local_size; - TRACE_ERR("In MPIDI_Comm_connect - port_name=%s rank=%d root=%d\n", port_name, rank, root); - - WORLDINTCOMMCNTR += 1; - comm_cntr = WORLDINTCOMMCNTR; - lcomm_cntr = WORLDINTCOMMCNTR; - - if (rank == root) - { - /* Establish a communicator to communicate with the root on the - other side. */ - mpi_errno = MPIDI_Create_inter_root_communicator_connect( - port_name, &tmp_comm, &new_vc); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_Create_inter_root_communicator_connect returned mpi_errno=%d\n", mpi_errno); - } - TRACE_ERR("after MPIDI_Create_inter_root_communicator_connect - tmp_comm=%p new_vc=%p mpi_errno=%d\n", tmp_comm, new_vc, mpi_errno); - - /* Make an array to translate local ranks to process group index - and rank */ - local_translation = MPL_malloc(local_comm_size*sizeof(pg_translation)); -/* MPIR_CHKLMEM_MALLOC(local_translation,pg_translation*, - local_comm_size*sizeof(pg_translation), - mpi_errno,"local_translation"); */ - - /* Make a list of the local communicator's process groups and encode - them in strings to be sent to the other side. - The encoded string for each process group contains the process - group id, size and all its KVS values */ - mpi_errno = MPIDI_ExtractLocalPGInfo( comm_ptr, local_translation, - &pg_list, &n_local_pgs ); - - /* Send the remote root: n_local_pgs, local_comm_size, - Recv from the remote root: n_remote_pgs, remote_comm_size, - recvcontext_id for newcomm */ - - send_ints[0] = n_local_pgs; - send_ints[1] = local_comm_size; - send_ints[2] = recvcontext_id; - - TRACE_ERR("connect:sending 3 ints, %d, %d, %d, and receiving 2 ints with sendtag=%d recvtag=%d\n", send_ints[0], send_ints[1], send_ints[2], sendtag, recvtag); - mpi_errno = MPIC_Sendrecv(send_ints, 3, MPI_INT, 0, - sendtag++, recv_ints, 3, MPI_INT, - 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - /* this is a no_port error because we may fail to connect - on the send if the port name is invalid */ - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } - - mpi_errno = MPIC_Sendrecv_replace(&comm_cntr, 1, MPI_LONG_LONG_INT, 0, - sendtag++, 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - /* this is a no_port error because we may fail to connect - on the send if the port name is invalid */ - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } - } - - /* broadcast the received info to local processes */ - TRACE_ERR("accept:broadcasting 2 ints - %d and %d\n", recv_ints[0], recv_ints[1]); - mpi_errno = MPIR_Bcast_intra(recv_ints, 3, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - - mpi_errno = MPIR_Bcast_intra(&comm_cntr, 1, MPI_LONG_LONG_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - - if(lcomm_cntr > comm_cntr) comm_cntr = lcomm_cntr; - - /* check if root was unable to connect to the port */ - - n_remote_pgs = recv_ints[0]; - remote_comm_size = recv_ints[1]; - context_id = recv_ints[2]; - - TRACE_ERR("MPIDI_Comm_connect - n_remote_pgs=%d remote_comm_size=%d context_id=%d\n", n_remote_pgs, - remote_comm_size, context_id); - remote_pg = MPL_malloc(n_remote_pgs * sizeof(MPIDI_PG_t*)); - remote_translation = MPL_malloc(remote_comm_size * sizeof(pg_translation)); - /* Exchange the process groups and their corresponding KVSes */ - if (rank == root) - { - mpi_errno = MPIDI_SendPGtoPeerAndFree( tmp_comm, &sendtag, pg_list ); - mpi_errno = MPIDI_ReceivePGAndDistribute( tmp_comm, comm_ptr, root, &recvtag, - n_remote_pgs, remote_pg ); - /* Receive the translations from remote process rank to process group - index */ - mpi_errno = MPIC_Sendrecv(local_translation, local_comm_size * 3, - MPI_INT, 0, sendtag++, - remote_translation, remote_comm_size * 3, - MPI_INT, 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno) { - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } - - for (i=0; i<remote_comm_size; i++) - { - TRACE_ERR(" remote_translation[%d].pg_index = %d\n remote_translation[%d].pg_rank = %d\n", - i, remote_translation[i].pg_index, i, remote_translation[i].pg_rank); - } - } - else - { - mpi_errno = MPIDI_ReceivePGAndDistribute( tmp_comm, comm_ptr, root, &recvtag, - n_remote_pgs, remote_pg ); - } - - /* Broadcast out the remote rank translation array */ - mpi_errno = MPIR_Bcast_intra(remote_translation, remote_comm_size * 3, MPI_INT, - root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - - char *pginfo = MPL_malloc(256*sizeof(char)); - memset(pginfo, 0, 256); - char cp[20]; - for (i=0; i<remote_comm_size; i++) - { - TRACE_ERR(" remote_translation[%d].pg_index = %d remote_translation[%d].pg_rank = %d remote_translation[%d].pg_taskid=%d\n", - i, remote_translation[i].pg_index, i, remote_translation[i].pg_rank, i, remote_translation[i].pg_taskid); - TRACE_ERR("remote_pg[remote_translation[%d].pg_index]->id=%s\n",i, (char *)(remote_pg[remote_translation[i].pg_index]->id)); - strcat(pginfo, (char *)(remote_pg[remote_translation[i].pg_index]->id)); - sprintf(cp, ":%d ", remote_translation[i].pg_taskid); - strcat(pginfo, cp); - - - } - pginfo[strlen(pginfo)]='\0'; - TRACE_ERR("connection info %s\n", pginfo); - /*MPIDI_Parse_connection_info(n_remote_pgs, remote_pg);*/ - MPL_free(pginfo); - - mpi_errno = MPIR_Comm_create(newcomm); - if (mpi_errno) TRACE_ERR("MPIR_Comm_create returned with mpi_errno=%d\n", mpi_errno); - - (*newcomm)->context_id = context_id; - (*newcomm)->recvcontext_id = recvcontext_id; - (*newcomm)->is_low_group = 1; - - mpi_errno = MPIDI_SetupNewIntercomm( comm_ptr, remote_comm_size, - remote_translation, n_remote_pgs, remote_pg, *newcomm ); - (*newcomm)->mpid.world_intercomm_cntr = comm_cntr; - WORLDINTCOMMCNTR = comm_cntr; - MPIDI_add_new_tranid(comm_cntr); - -/* MPIDI_Parse_connection_info(n_remote_pgs, remote_pg); */ - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_SetupNewIntercomm returned with mpi_errno=%d\n", mpi_errno); - } - - /* synchronize with remote root */ - if (rank == root) - { - mpi_errno = MPIC_Sendrecv(&i, 0, MPI_INT, 0, - sendtag++, &j, 0, MPI_INT, - 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } - - /* All communication with remote root done. Release the communicator. */ - MPIR_Comm_release(tmp_comm,0); - } - - TRACE_ERR("connect:barrier\n"); - mpi_errno = MPIR_Barrier_intra(comm_ptr, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Barrier_intra returned with mpi_errno=%d\n", mpi_errno); - } - - TRACE_ERR("connect:free new vc\n"); - -fn_exit: - if(local_translation) MPL_free(local_translation); - return mpi_errno; - -fn_fail: - goto fn_exit; -} - -/* - * Extract all of the process groups from the given communicator and - * form a list (returned in pg_list) of those process groups. - * Also returned is an array (local_translation) that contains tuples mapping - * rank in process group to rank in that communicator (local translation - * must be allocated before this routine is called). The number of - * distinct process groups is returned in n_local_pgs_p . - * - * This allows an intercomm_create to exchange the full description of - * all of the process groups that have made up the communicator that - * will define the "remote group". - */ -static int MPIDI_ExtractLocalPGInfo( struct MPIR_Comm *comm_p, - pg_translation local_translation[], - pg_node **pg_list_p, - int *n_local_pgs_p ) -{ - pg_node *pg_list = 0, *pg_iter, *pg_trailer; - int i, cur_index = 0, local_comm_size, mpi_errno = 0; - char *pg_id; - - local_comm_size = comm_p->local_size; - - /* Make a list of the local communicator's process groups and encode - them in strings to be sent to the other side. - The encoded string for each process group contains the process - group id, size and all its KVS values */ - - cur_index = 0; - pg_list = MPL_malloc(sizeof(pg_node)); - - pg_list->pg_id = MPL_strdup(comm_p->vcr[0]->pg->id); - pg_list->index = cur_index++; - pg_list->next = NULL; - /* XXX DJG FIXME-MT should we be checking this? the add/release macros already check this */ - TRACE_ERR("MPIR_Object_get_ref(comm_p->vcr[0]->pg) comm_p=%x vsr=%x pg=%x %d\n", comm_p, comm_p->vcr[0], comm_p->vcr[0]->pg, MPIR_Object_get_ref(comm_p->vcr[0]->pg)); - MPIR_Assert( MPIR_Object_get_ref(comm_p->vcr[0]->pg)); - mpi_errno = MPIDI_PG_To_string(comm_p->vcr[0]->pg, &pg_list->str, - &pg_list->lenStr ); - TRACE_ERR("pg_list->str=%s pg_list->lenStr=%d\n", pg_list->str, pg_list->lenStr); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_PG_To_string returned with mpi_errno=%d\n", mpi_errno); - } - TRACE_ERR("PG as string is %s\n", pg_list->str ); - local_translation[0].pg_index = 0; - local_translation[0].pg_rank = comm_p->vcr[0]->pg_rank; - local_translation[0].pg_taskid = comm_p->vcr[0]->taskid; - TRACE_ERR("local_translation[0].pg_index=%d local_translation[0].pg_rank=%d\n", local_translation[0].pg_index, local_translation[0].pg_rank); - pg_iter = pg_list; - for (i=1; i<local_comm_size; i++) { - pg_iter = pg_list; - pg_trailer = pg_list; - while (pg_iter != NULL) { - /* Check to ensure pg is (probably) valid */ - /* XXX DJG FIXME-MT should we be checking this? the add/release macros already check this */ - MPIR_Assert(MPIR_Object_get_ref(comm_p->vcr[i]->pg) != 0); - if (MPIDI_PG_Id_compare(comm_p->vcr[i]->pg->id, pg_iter->pg_id)) { - local_translation[i].pg_index = pg_iter->index; - local_translation[i].pg_rank = comm_p->vcr[i]->pg_rank; - local_translation[i].pg_taskid = comm_p->vcr[i]->taskid; - TRACE_ERR("local_translation[%d].pg_index=%d local_translation[%d].pg_rank=%d\n", i, local_translation[i].pg_index, i,local_translation[i].pg_rank); - break; - } - if (pg_trailer != pg_iter) - pg_trailer = pg_trailer->next; - pg_iter = pg_iter->next; - } - if (pg_iter == NULL) { - /* We use MPL_malloc directly because we do not know in - advance how many nodes we may allocate */ - pg_iter = (pg_node*)MPL_malloc(sizeof(pg_node)); - pg_iter->pg_id = MPL_strdup(comm_p->vcr[i]->pg->id); - pg_iter->index = cur_index++; - pg_iter->next = NULL; - mpi_errno = MPIDI_PG_To_string(comm_p->vcr[i]->pg, &pg_iter->str, - &pg_iter->lenStr ); - - TRACE_ERR("cur_index=%d pg_iter->str=%s pg_iter->lenStr=%d\n", cur_index, pg_iter->str, pg_iter->lenStr); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_PG_To_string returned with mpi_errno=%d\n", mpi_errno); - } - local_translation[i].pg_index = pg_iter->index; - local_translation[i].pg_rank = comm_p->vcr[i]->pg_rank; - local_translation[i].pg_taskid = comm_p->vcr[i]->taskid; - pg_trailer->next = pg_iter; - } - } - - *n_local_pgs_p = cur_index; - *pg_list_p = pg_list; - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - - - -/* The root process in comm_ptr receives strings describing the - process groups and then distributes them to the other processes - in comm_ptr. - See SendPGToPeer for the routine that sends the descriptions */ -static int MPIDI_ReceivePGAndDistribute( struct MPIR_Comm *tmp_comm, struct MPIR_Comm *comm_ptr, - int root, int *recvtag_p, - int n_remote_pgs, MPIDI_PG_t *remote_pg[] ) -{ - char *pg_str = 0; - char *pginfo = 0; - int i, j, flag; - int rank = comm_ptr->rank; - int mpi_errno = 0; - int recvtag = *recvtag_p; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - - TRACE_ERR("MPIDI_ReceivePGAndDistribute - n_remote_pgs=%d\n", n_remote_pgs); - for (i=0; i<n_remote_pgs; i++) { - - if (rank == root) { - /* First, receive the pg description from the partner */ - mpi_errno = MPIC_Recv(&j, 1, MPI_INT, 0, recvtag++, - tmp_comm, MPI_STATUS_IGNORE, &errflag); - *recvtag_p = recvtag; - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Recv returned with mpi_errno=%d\n", mpi_errno); - } - pg_str = (char*)MPL_malloc(j); - mpi_errno = MPIC_Recv(pg_str, j, MPI_CHAR, 0, recvtag++, - tmp_comm, MPI_STATUS_IGNORE, &errflag); - *recvtag_p = recvtag; - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Recv returned with mpi_errno=%d\n", mpi_errno); - } - } - - /* Broadcast the size and data to the local communicator */ - TRACE_ERR("accept:broadcasting 1 int\n"); - mpi_errno = MPIR_Bcast_intra(&j, 1, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno != MPI_SUCCESS) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - - if (rank != root) { - /* The root has already allocated this string */ - pg_str = (char*)MPL_malloc(j); - } - TRACE_ERR("accept:broadcasting string of length %d\n", j); - pg_str[j-1]='\0'; - mpi_errno = MPIR_Bcast_intra(pg_str, j, MPI_CHAR, root, comm_ptr, &errflag); - if (mpi_errno != MPI_SUCCESS) - TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - /* Then reconstruct the received process group. This step - also initializes the created process group */ - - TRACE_ERR("Adding connection information - pg_str=%s\n", pg_str); - TRACE_ERR("Creating pg from string %s flag=%d\n", pg_str, flag); - mpi_errno = MPIDI_PG_Create_from_string(pg_str, &remote_pg[i], &flag); - TRACE_ERR("remote_pg[%d]->id=%s\n", i, (char*)(remote_pg[i]->id)); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_PG_Create_from_string returned with mpi_errno=%d\n", mpi_errno); - } - - MPL_free(pg_str); - } - /*MPIDI_Parse_connection_info(pg_str); */ - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -void MPIDI_Add_connection_info(int wid, int wsize, pami_task_t *taskids) { - int jobIdSize=64; - char jobId[jobIdSize]; - int ref_count, i; - conn_info *tmp_node1=NULL, *tmp_node2=NULL; - - TRACE_ERR("MPIDI_Add_connection_info ENTER wid=%d wsize=%d\n", wid, wsize); - PMI2_Job_GetId(jobId, jobIdSize); - if(atoi(jobId) == wid) - return; - - /* FIXME: check the lock */ - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - if(_conn_info_list == NULL) { /* Connection list is not yet created */ - _conn_info_list = (conn_info*) MPL_malloc(sizeof(conn_info)); - _conn_info_list->rem_world_id = wid; - _conn_info_list->ref_count = 1; - - ref_count = _conn_info_list->ref_count; - if(taskids != NULL) { - _conn_info_list->rem_taskids = MPL_malloc((wsize+1)*sizeof(int)); - for(i=0;i<wsize;i++) { - _conn_info_list->rem_taskids[i] = taskids[i]; - } - _conn_info_list->rem_taskids[i] = -1; - } - else - _conn_info_list->rem_taskids = NULL; - _conn_info_list->next = NULL; - } - else { - tmp_node1 = _conn_info_list; - while(tmp_node1) { - tmp_node2 = tmp_node1; - if(tmp_node1->rem_world_id == wid) - break; - tmp_node1 = tmp_node1->next; - } - if(tmp_node1) { /* Connection already exists. Increment reference count */ - if(tmp_node1->ref_count == 0) { - if(taskids != NULL) { - tmp_node1->rem_taskids = MPL_malloc((wsize+1)*sizeof(int)); - for(i=0;i<wsize;i++) { - tmp_node1->rem_taskids[i] = taskids[i]; - } - tmp_node1->rem_taskids[i] = -1; - } - tmp_node1->rem_world_id = wid; - } - tmp_node1->ref_count++; - ref_count = tmp_node1->ref_count; - } - else { /* Connection do not exists. Create a new connection */ - tmp_node2->next = (conn_info*) MPL_malloc(sizeof(conn_info)); - tmp_node2 = tmp_node2->next; - tmp_node2->rem_world_id = wid; - tmp_node2->ref_count = 1; - - ref_count = tmp_node2->ref_count; - if(taskids != NULL) { - tmp_node2->rem_taskids = MPL_malloc((wsize+1)*sizeof(int)); - for(i=0;i<wsize;i++) { - tmp_node2->rem_taskids[i] = taskids[i]; - } - tmp_node2->rem_taskids[i] = -1; - } - else - tmp_node2->rem_taskids = NULL; - tmp_node2->next = NULL; - } - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - - tmp_node1 = _conn_info_list; - while(tmp_node1) { - TRACE_ERR("REM WORLD=%d ref_count=%d", tmp_node1->rem_world_id,tmp_node1->ref_count); - - tmp_node1 = tmp_node1->next; - } -} - - -/** - * This routine adds the remote world (wid) to local known world linked list - * if there is no record of it before, or increment the reference count - * associated with world (wid) if it is known before - */ -void MPIDI_Parse_connection_info(int n_remote_pgs, MPIDI_PG_t **remote_pg) { - int i, p, ref_count=0; - int jobIdSize=8; - char jobId[jobIdSize]; - char *pginfo_sav, *pgid_taskid_sav, *pgid, *pgid_taskid[20], *pginfo_tmp, *cp3, *cp2; - pami_task_t *taskids; - int n_rem_wids=0; - int mpi_errno = MPI_SUCCESS; - MPIDI_PG_t *existing_pg; - - for(p=0; p<n_remote_pgs; p++) { - TRACE_ERR("call MPIDI_PG_Find to find %s\n", (char*)(remote_pg[p]->id)); - mpi_errno = MPIDI_PG_Find(remote_pg[p]->id, &existing_pg); - if (mpi_errno) TRACE_ERR("MPIDI_PG_Find failed\n"); - - if (existing_pg != NULL) { - taskids = MPL_malloc((existing_pg->size)*sizeof(pami_task_t)); - for(i=0; i<existing_pg->size; i++) { - taskids[i]=existing_pg->vct[i].taskid; - TRACE_ERR("id=%s taskids[%d]=%d\n", (char*)(remote_pg[p]->id), i, taskids[i]); - } - MPIDI_Add_connection_info(atoi((char*)(remote_pg[p]->id)), existing_pg->size, taskids); - MPL_free(taskids); - } - } -} - - - -/** - * Function to increment the active message counter for a particular trasaction id. - * This function is used inside disconnect routine - * whichAM = FIRST_AM/SECOND_AM/LAST_AM - */ -void MPIDI_increment_AM_cntr_for_tranid(long long tranid, int whichAM) -{ - transactionID_struct *tridtmp; - - /* No error thrown here if tranid not found. This is for the case where timout - * happened in MPI_Comm_disconnect and tasks have freed the tranid list node - * and now after this the Active message is received. - */ - - tridtmp = _transactionID_list; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - while(tridtmp != NULL) { - if(tridtmp->tranid == tranid) { - tridtmp->cntr_for_AM[whichAM]++; - break; - } - tridtmp = tridtmp->next; - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - - TRACE_ERR("MPIDI_increment_AM_cntr_for_tranid - tridtmp->cntr_for_AM[%d]=%d\n", - whichAM, tridtmp->cntr_for_AM[whichAM]); -} - -/** - * Function to free a partucular trasaction id node from the trasaction id list. - * This function is called inside disconnect routine once the remote connection is - * terminated - */ -void MPIDI_free_tranid_node(long long tranid) -{ - transactionID_struct *tridtmp, *tridtmp2; - - MPID_assert(_transactionID_list != NULL); - - tridtmp = tridtmp2 = _transactionID_list; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - while(tridtmp != NULL) { - if(tridtmp->tranid == tranid) { - /* If there is only one node */ - if(_transactionID_list->next == NULL) { - MPL_free(_transactionID_list->cntr_for_AM); - MPL_free(_transactionID_list); - _transactionID_list = NULL; - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - return; - } - /* If more than one node and if this is the first node of the list */ - if(tridtmp == _transactionID_list && tridtmp->next != NULL) { - _transactionID_list = _transactionID_list->next; - MPL_free(tridtmp->cntr_for_AM); - MPL_free(tridtmp); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - return; - } - /* For rest all other nodes position of the list */ - tridtmp2->next = tridtmp->next; - MPL_free(tridtmp->cntr_for_AM); - MPL_free(tridtmp); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - return; - } - tridtmp2 = tridtmp; - tridtmp = tridtmp->next; - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); -} - -/** This routine is used inside finalize to free all the nodes - * if the disconnect call has not been called - */ -void MPIDI_free_all_tranid_node() -{ - transactionID_struct *tridtmp; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - while(_transactionID_list != NULL) { - tridtmp = _transactionID_list; - _transactionID_list = _transactionID_list->next; - MPL_free(tridtmp->cntr_for_AM); - MPL_free(tridtmp); - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); -} - -/* Sends the process group information to the peer and frees the - pg_list */ -static int MPIDI_SendPGtoPeerAndFree( struct MPIR_Comm *tmp_comm, int *sendtag_p, - pg_node *pg_list ) -{ - int mpi_errno = 0; - int sendtag = *sendtag_p, i; - pg_node *pg_iter; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - - while (pg_list != NULL) { - pg_iter = pg_list; - i = pg_iter->lenStr; - TRACE_ERR("connect:sending 1 int: %d\n", i); - mpi_errno = MPIC_Send(&i, 1, MPI_INT, 0, sendtag++, tmp_comm, &errflag); - *sendtag_p = sendtag; - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Send returned with mpi_errno=%d\n", mpi_errno); - } - - TRACE_ERR("connect:sending string length %d\n", i); - mpi_errno = MPIC_Send(pg_iter->str, i, MPI_CHAR, 0, sendtag++, - tmp_comm, &errflag); - *sendtag_p = sendtag; - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Send returned with mpi_errno=%d\n", mpi_errno); - } - - pg_list = pg_list->next; - MPL_free(pg_iter->str); - MPL_free(pg_iter->pg_id); - MPL_free(pg_iter); - } - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -/* ---------------------------------------------------------------------- */ -/* - * MPIDI_Comm_accept() - - Algorithm: First dequeue the vc from the accept queue (it was - enqueued by the progress engine in response to a connect request - from the root process that is attempting the connection on - the connect side). Use this vc to create an - intercommunicator between this root and the root on the connect - side. Use this intercomm. to communicate the other information - needed to create the real intercommunicator between the processes - on the two sides. Then free the intercommunicator between the - roots. Most of the complexity is because there can be multiple - process groups on each side. - - */ -int MPIDI_Comm_accept(const char *port_name, MPIR_Info *info, int root, - struct MPIR_Comm *comm_ptr, struct MPIR_Comm **newcomm) -{ - int mpi_errno=MPI_SUCCESS; - int i, j, rank, recv_ints[3], send_ints[3], context_id; - int remote_comm_size=0; - struct MPIR_Comm *tmp_comm = NULL, *intercomm; - MPID_VCR new_vc = NULL; - int sendtag=100, recvtag=100, local_comm_size; - int n_local_pgs=1, n_remote_pgs; - pg_translation *local_translation = NULL, *remote_translation = NULL; - pg_node *pg_list = NULL; - MPIDI_PG_t **remote_pg = NULL; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - char send_char[16], recv_char[16], remote_taskids[16]; - long long comm_cntr, lcomm_cntr; - int leader_taskid; - - /* Create the new intercommunicator here. We need to send the - context id to the other side. */ - mpi_errno = MPIR_Comm_create(newcomm); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Comm_create returned with mpi_errno=%d\n", mpi_errno); - } - mpi_errno = MPIR_Get_contextid( comm_ptr, &(*newcomm)->recvcontext_id ); - TRACE_ERR("In MPIDI_Comm_accept - MPIR_Get_contextid=%d\n", (*newcomm)->recvcontext_id); - if (mpi_errno) TRACE_ERR("MPIR_Get_contextid returned with mpi_errno=%d\n", mpi_errno); - /* FIXME why is this commented out? */ - /* (*newcomm)->context_id = (*newcomm)->recvcontext_id; */ - - rank = comm_ptr->rank; - local_comm_size = comm_ptr->local_size; - - WORLDINTCOMMCNTR += 1; - comm_cntr = WORLDINTCOMMCNTR; - lcomm_cntr = WORLDINTCOMMCNTR; - - if (rank == root) - { - /* Establish a communicator to communicate with the root on the - other side. */ - mpi_errno = MPIDI_Create_inter_root_communicator_accept(port_name, - &tmp_comm, &new_vc); - TRACE_ERR("done MPIDI_Create_inter_root_communicator_accept mpi_errno=%d tmp_comm=%p new_vc=%p \n", mpi_errno, tmp_comm, new_vc); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_Create_inter_root_communicator_accept returned with mpi_errno=%d\n", mpi_errno); - } - - /* Make an array to translate local ranks to process group index and - rank */ - local_translation = MPL_malloc(local_comm_size*sizeof(pg_translation)); -/* MPIR_CHKLMEM_MALLOC(local_translation,pg_translation*, - local_comm_size*sizeof(pg_translation), - mpi_errno,"local_translation"); */ - - /* Make a list of the local communicator's process groups and encode - them in strings to be sent to the other side. - The encoded string for each process group contains the process - group id, size and all its KVS values */ - mpi_errno = MPIDI_ExtractLocalPGInfo( comm_ptr, local_translation, - &pg_list, &n_local_pgs ); - /* Send the remote root: n_local_pgs, local_comm_size, context_id for - newcomm. - Recv from the remote root: n_remote_pgs, remote_comm_size */ - - send_ints[0] = n_local_pgs; - send_ints[1] = local_comm_size; - send_ints[2] = (*newcomm)->recvcontext_id; - - TRACE_ERR("accept:sending 3 ints, %d, %d, %d, and receiving 2 ints with sendtag=%d recvtag=%d\n", send_ints[0], send_ints[1], send_ints[2], sendtag, recvtag); - mpi_errno = MPIC_Sendrecv(send_ints, 3, MPI_INT, 0, - sendtag++, recv_ints, 3, MPI_INT, - 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } -#if 0 - send_char = pg_list->str; - TRACE_ERR("accept:sending 1 string and receiving 1 string\n", send_char, recv_char); - mpi_errno = MPIC_Sendrecv(send_char, 1, MPI_CHAR, 0, - sendtag++, recv_char, 3, MPI_CHAR, - 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } -#endif - mpi_errno = MPIC_Sendrecv_replace(&comm_cntr, 1, MPI_LONG_LONG_INT, 0, - sendtag++, 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - /* this is a no_port error because we may fail to connect - on the send if the port name is invalid */ - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } - - } - - /* broadcast the received info to local processes */ - TRACE_ERR("accept:broadcasting 2 ints - %d and %d\n", recv_ints[0], recv_ints[1]); - mpi_errno = MPIR_Bcast_intra(recv_ints, 3, MPI_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - - mpi_errno = MPIR_Bcast_intra(&comm_cntr, 1, MPI_LONG_LONG_INT, root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - - if(lcomm_cntr > comm_cntr) comm_cntr = lcomm_cntr; - n_remote_pgs = recv_ints[0]; - remote_comm_size = recv_ints[1]; - context_id = recv_ints[2]; - remote_pg = MPL_malloc(n_remote_pgs * sizeof(MPIDI_PG_t*)); - remote_translation = MPL_malloc(remote_comm_size * sizeof(pg_translation)); - TRACE_ERR("[%d]accept:remote process groups: %d\nremote comm size: %d\nrecv_char: %s\n", rank, n_remote_pgs, remote_comm_size, remote_taskids); - - /* Exchange the process groups and their corresponding KVSes */ - if (rank == root) - { - /* The root receives the PG from the peer (in tmp_comm) and - distributes them to the processes in comm_ptr */ - mpi_errno = MPIDI_ReceivePGAndDistribute( tmp_comm, comm_ptr, root, &recvtag, - n_remote_pgs, remote_pg ); - - mpi_errno = MPIDI_SendPGtoPeerAndFree( tmp_comm, &sendtag, pg_list ); - - /* Receive the translations from remote process rank to process group index */ - TRACE_ERR("accept:sending %d ints and receiving %d ints\n", local_comm_size * 2, remote_comm_size * 2); - mpi_errno = MPIC_Sendrecv(local_translation, local_comm_size * 3, - MPI_INT, 0, sendtag++, - remote_translation, remote_comm_size * 3, - MPI_INT, 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - for (i=0; i<remote_comm_size; i++) - { - TRACE_ERR(" remote_translation[%d].pg_index = %d\n remote_translation[%d].pg_rank = %d\n", - i, remote_translation[i].pg_index, i, remote_translation[i].pg_rank); - } - } - else - { - mpi_errno = MPIDI_ReceivePGAndDistribute( tmp_comm, comm_ptr, root, &recvtag, - n_remote_pgs, remote_pg ); - } - for(i=0; i<n_remote_pgs; i++) - { - TRACE_ERR("after calling MPIDI_ReceivePGAndDistribute - remote_pg[%d]->id=%s\n",i, (char *)(remote_pg[i]->id)); - } - - - /* Broadcast out the remote rank translation array */ - TRACE_ERR("Broadcast remote_translation"); - mpi_errno = MPIR_Bcast_intra(remote_translation, remote_comm_size * 3, MPI_INT, - root, comm_ptr, &errflag); - if (mpi_errno) TRACE_ERR("MPIR_Bcast_intra returned with mpi_errno=%d\n", mpi_errno); - TRACE_ERR("[%d]accept:Received remote_translation after broadcast:\n", rank); - char *pginfo = MPL_malloc(256*sizeof(char)); - memset(pginfo, 0, 256); - char cp[20]; - for (i=0; i<remote_comm_size; i++) - { - TRACE_ERR(" remote_translation[%d].pg_index = %d remote_translation[%d].pg_rank = %d remote_translation[%d].pg_taskid=%d\n", - i, remote_translation[i].pg_index, i, remote_translation[i].pg_rank, i, remote_translation[i].pg_taskid); - TRACE_ERR("remote_pg[remote_translation[%d].pg_index]->id=%s\n",i, (char *)(remote_pg[remote_translation[i].pg_index]->id)); - strcat(pginfo, (char *)(remote_pg[remote_translation[i].pg_index]->id)); - sprintf(cp, ":%d ", remote_translation[i].pg_taskid); - strcat(pginfo, cp); - - - } - pginfo[strlen(pginfo)]='\0'; - TRACE_ERR("connection info %s\n", pginfo); -/* MPIDI_Parse_connection_info(n_remote_pgs, remote_pg); */ - MPL_free(pginfo); - - - /* Now fill in newcomm */ - intercomm = *newcomm; - intercomm->context_id = context_id; - intercomm->is_low_group = 0; - - mpi_errno = MPIDI_SetupNewIntercomm( comm_ptr, remote_comm_size, - remote_translation, n_remote_pgs, remote_pg, intercomm ); - intercomm->mpid.world_intercomm_cntr = comm_cntr; - WORLDINTCOMMCNTR = comm_cntr; - MPIDI_add_new_tranid(comm_cntr); - - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_SetupNewIntercomm returned with mpi_errno=%d\n", mpi_errno); - } - - /* synchronize with remote root */ - if (rank == root) - { - mpi_errno = MPIC_Sendrecv(&i, 0, MPI_INT, 0, - sendtag++, &j, 0, MPI_INT, - 0, recvtag++, tmp_comm, - MPI_STATUS_IGNORE, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIC_Sendrecv returned with mpi_errno=%d\n", mpi_errno); - } - - /* All communication with remote root done. Release the communicator. */ - MPIR_Comm_release(tmp_comm,0); - } - - mpi_errno = MPIR_Barrier_intra(comm_ptr, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Barrier_intra returned with mpi_errno=%d\n", mpi_errno); - } - -fn_exit: - if(local_translation) MPL_free(local_translation); - return mpi_errno; - -fn_fail: - goto fn_exit; -} - -/* ------------------------------------------------------------------------- */ - -/* This routine initializes the new intercomm, setting up the - VCRT and other common structures. The is_low_group and context_id - fields are NOT set because they differ in the use of this - routine in Comm_accept and Comm_connect. The virtual connections - are initialized from a collection of process groups. - - Input parameters: -+ comm_ptr - communicator that gives the group for the "local" group on the - new intercommnicator -. remote_comm_size - size of remote group -. remote_translation - array that specifies the process group and rank in - that group for each of the processes to include in the remote group of the - new intercommunicator -- remote_pg - array of remote process groups - - Input/Output Parameter: -. intercomm - New intercommunicator. The intercommunicator must already - have been allocated; this routine initializes many of the fields - - Note: - This routine performance a barrier over 'comm_ptr'. Why? -*/ -static int MPIDI_SetupNewIntercomm( struct MPIR_Comm *comm_ptr, int remote_comm_size, - pg_translation remote_translation[], - int n_remote_pgs, MPIDI_PG_t **remote_pg, - struct MPIR_Comm *intercomm ) -{ - int mpi_errno = MPI_SUCCESS, i, j, index=0; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - int total_rem_world_cnts, p=0; - char *world_tasks, *cp1; - conn_info *tmp_node; - int conn_world_ids[64]; - MPID_VCR *worldlist; - int worldsize; - pami_endpoint_t dest; - MPIR_Comm *comm; - pami_task_t leader1=-1, leader2=-1, leader_taskid=-1; - long long comm_cntr=0, lcomm_cntr=-1; - int jobIdSize=64; - char jobId[jobIdSize]; - - TRACE_ERR("MPIDI_SetupNewIntercomm - remote_comm_size=%d\n", remote_comm_size); - /* FIXME: How much of this could/should be common with the - upper level (src/mpi/comm/ *.c) code? For best robustness, - this should use the same routine (not copy/paste code) as - in the upper level code. */ - intercomm->attributes = NULL; - intercomm->remote_size = remote_comm_size; - intercomm->local_size = comm_ptr->local_size; - intercomm->rank = comm_ptr->rank; - intercomm->local_group = NULL; - intercomm->remote_group = NULL; - intercomm->comm_kind = MPIR_COMM_KIND__INTERCOMM; - intercomm->local_comm = NULL; - intercomm->coll_fns = NULL; - intercomm->mpid.world_ids = NULL; /*FIXME*/ - - /* Point local vcr, vcrt at those of incoming intracommunicator */ - intercomm->local_vcrt = comm_ptr->vcrt; - MPID_VCRT_Add_ref(comm_ptr->vcrt); - intercomm->local_vcr = comm_ptr->vcr; - for(i=0; i<comm_ptr->local_size; i++) - TRACE_ERR("intercomm->local_vcr[%d]->pg_rank=%d comm_ptr->vcr[%d].pg_rank=%d intercomm->local_vcr[%d]->taskid=%d comm_ptr->vcr[%d]->taskid=%d\n", i, intercomm->local_vcr[i]->pg_rank, i, comm_ptr->vcr[i]->pg_rank, i, intercomm->local_vcr[i]->taskid, i, comm_ptr->vcr[i]->taskid); - - /* Set up VC reference table */ - mpi_errno = MPID_VCRT_Create(intercomm->remote_size, &intercomm->vcrt); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPID_VCRT_Create returned with mpi_errno=%d\n", mpi_errno); - } - mpi_errno = MPID_VCRT_Get_ptr(intercomm->vcrt, &intercomm->vcr); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPID_VCRT_Get_ptr returned with mpi_errno=%d\n", mpi_errno); - } - - for (i=0; i < intercomm->remote_size; i++) { - MPIDI_PG_Dup_vcr(remote_pg[remote_translation[i].pg_index], - remote_translation[i].pg_rank, remote_translation[i].pg_taskid,&intercomm->vcr[i]); - TRACE_ERR("MPIDI_SetupNewIntercomm - pg_id=%s pg_rank=%d pg_taskid=%d intercomm->vcr[%d]->taskid=%d intercomm->vcr[%d]->pg=%x\n ", remote_pg[remote_translation[i].pg_index]->id, remote_translation[i].pg_rank, remote_translation[i].pg_taskid, i, intercomm->vcr[i]->taskid, i, intercomm->vcr[i]->pg); - PAMI_Endpoint_create(MPIDI_Client, remote_translation[i].pg_taskid, 0, &dest); - PAMI_Resume(MPIDI_Context[0], - &dest, 1); - } - - MPIDI_Parse_connection_info(n_remote_pgs, remote_pg); - - /* anchor connection information in mpid */ - total_rem_world_cnts = 0; - tmp_node = _conn_info_list; - p=0; - while(tmp_node != NULL) { - total_rem_world_cnts++; - conn_world_ids[p++]=tmp_node->rem_world_id; - tmp_node = tmp_node->next; - } - if(intercomm->mpid.world_ids) { /* need to look at other places that may populate world id list for this communicator */ - for(i=0;intercomm->mpid.world_ids[i]!=-1;i++) - { - for(j=0;j<total_rem_world_cnts;j++) { - if(intercomm->mpid.world_ids[i] == conn_world_ids[j]) { - conn_world_ids[j] = -1; - } - } - } - /* Now Total world_ids inside intercomm->world_ids = i, excluding last entry of ' -1' */ - index = 0; - for(j=0;j<total_rem_world_cnts;j++) { - if(conn_world_ids[j] != -1) - index++; - } - if(index) { - intercomm->mpid.world_ids = MPL_malloc((index+i+1)*sizeof(int)); - /* Current index i inside intercomm->mpid.world_ids is - * the place where next world_id can be added - */ - for(j=0;j<total_rem_world_cnts;j++) { - if(conn_world_ids[j] != -1) { - intercomm->mpid.world_ids[i++] = conn_world_ids[j]; - } - } - intercomm->mpid.world_ids[i] = -1; - } - } - else { - index=0; - intercomm->mpid.world_ids = MPL_malloc((n_remote_pgs+1)*sizeof(int)); - PMI2_Job_GetId(jobId, jobIdSize); - for(i=0;i<n_remote_pgs;i++) { - if(atoi(jobId) != atoi((char *)remote_pg[i]->id) ) - intercomm->mpid.world_ids[index++] = atoi((char *)remote_pg[i]->id); - } - intercomm->mpid.world_ids[index++] = -1; - } - for(i=0; intercomm->mpid.world_ids[i] != -1; i++) - TRACE_ERR("intercomm=%x intercomm->mpid.world_ids[%d]=%d\n", intercomm, i, intercomm->mpid.world_ids[i]); - - leader_taskid = comm_ptr->vcr[0]->taskid; - - MPIR_Comm *comm_world_ptr = MPIR_Process.comm_world; - worldlist = comm_world_ptr->vcr; - worldsize = comm_world_ptr->local_size; - comm = intercomm; - for(i=0;i<intercomm->local_size;i++) - { - for(j=0;j<comm_world_ptr->local_size;j++) - { - if(intercomm->local_vcr[i]->taskid == comm_world_ptr->vcr[j]->taskid) { - leader1 = comm_world_ptr->vcr[j]->taskid; - break; - } - } - if(leader1 != -1) - break; - } - for(i=0;i<intercomm->remote_size;i++) - { - for(j=0;j<comm_world_ptr->local_size;j++) - { - if(intercomm->vcr[i]->taskid == comm_world_ptr->vcr[j]->taskid) { - leader2 = comm_world_ptr->vcr[j]->taskid; - break; - } - } - if(leader2 != -1) - break; - } - - if(leader1 == -1) - leader_taskid = leader2; - else if(leader2 == -1) - leader_taskid = leader1; - else - leader_taskid = leader1 < leader2 ? leader1 : leader2; - intercomm->mpid.local_leader = leader_taskid; - - mpi_errno = MPIR_Comm_commit(intercomm); - if (mpi_errno) TRACE_ERR("MPIR_Comm_commit returned with mpi_errno=%d\n", mpi_errno); - - mpi_errno = MPIR_Barrier_intra(comm_ptr, &errflag); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIR_Barrier_intra returned with mpi_errno=%d\n", mpi_errno); - } - - fn_exit: - if(remote_pg) MPL_free(remote_pg); - if(remote_translation) MPL_free(remote_translation); - return mpi_errno; - - fn_fail: - goto fn_exit; -} - - -/* Attempt to dequeue a vc from the accept queue. If the queue is - empty or the port_name_tag doesn't match, return a NULL vc. */ -int MPIDI_Acceptq_dequeue(MPID_VCR * vcr, int port_name_tag) -{ - int mpi_errno=MPI_SUCCESS; - MPIDI_Acceptq_t *q_item, *prev; - *vcr = NULL; - q_item = acceptq_head; - prev = q_item; - - while (q_item != NULL) - { - if (q_item->port_name_tag == port_name_tag) - { - *vcr = q_item->vcr; - - if ( q_item == acceptq_head ) - acceptq_head = q_item->next; - else - prev->next = q_item->next; - - MPL_free(q_item); - AcceptQueueSize--; - break;; - } - else - { - prev = q_item; - q_item = q_item->next; - } - } - - return mpi_errno; -} - - -/** - * This routine return the list of taskids associated with world (wid) - */ -int* MPIDI_get_taskids_in_world_id(int wid) { - conn_info *tmp_node; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - tmp_node = _conn_info_list; - while(tmp_node != NULL) { - if(tmp_node->rem_world_id == wid) { - break; - } - tmp_node = tmp_node->next; - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - if(tmp_node == NULL) - return NULL; - else - return (tmp_node->rem_taskids); -} - -void MPIDI_IpState_reset(int dest) -{ - MPIDI_In_cntr_t *in_cntr; - in_cntr=&MPIDI_In_cntr[dest]; - - in_cntr->n_OutOfOrderMsgs = 0; - in_cntr->nMsgs = 0; - in_cntr->OutOfOrderList = NULL; -} - - -void MPIDI_OpState_reset(int dest) -{ - MPIDI_Out_cntr_t *out_cntr; - out_cntr=&MPIDI_Out_cntr[dest]; - - out_cntr->nMsgs = 0; - out_cntr->unmatched = 0; -} - - -/** - * This routine return the connection reference count associated with the - * remote world identified by wid - */ -int MPIDI_get_refcnt_of_world(int wid) { - conn_info *tmp_node; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - tmp_node = _conn_info_list; - while(tmp_node != NULL) { - if(tmp_node->rem_world_id == wid) { - break; - } - tmp_node = tmp_node->next; - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - if(tmp_node == NULL) - return 0; - else - return (tmp_node->ref_count); -} - -/** - * This routine delete world (wid) from linked list of known world descriptors - */ -void MPIDI_delete_conn_record(int wid) { - conn_info *tmp_node1, *tmp_node2; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - tmp_node1 = tmp_node2 = _conn_info_list; - while(tmp_node1) { - if(tmp_node1->rem_world_id == wid) { - if(tmp_node1 == tmp_node2) { - _conn_info_list = tmp_node1->next; - } - else { - tmp_node2->next = tmp_node1->next; - } - if(tmp_node1->rem_taskids != NULL) - MPL_free(tmp_node1->rem_taskids); - MPL_free(tmp_node1); - break; - } - tmp_node2 = tmp_node1; - tmp_node1 = tmp_node1->next; - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); -} - - -int MPID_PG_BCast( MPIR_Comm *peercomm_p, MPIR_Comm *comm_p, int root ) -{ - int n_local_pgs=0, mpi_errno = MPI_SUCCESS; - pg_translation *local_translation = 0; - pg_node *pg_list, *pg_next, *pg_head = 0; - int rank, i, peer_comm_size; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - MPIR_CHKLMEM_DECL(1); - - peer_comm_size = comm_p->local_size; - rank = comm_p->rank; - - local_translation = (pg_translation*)MPL_malloc(peer_comm_size*sizeof(pg_translation)); - if (rank == root) { - /* Get the process groups known to the *peercomm* */ - MPIDI_ExtractLocalPGInfo( peercomm_p, local_translation, &pg_head, - &n_local_pgs ); - } - - /* Now, broadcast the number of local pgs */ - mpi_errno = MPIR_Bcast_impl( &n_local_pgs, 1, MPI_INT, root, comm_p, &errflag); - - pg_list = pg_head; - for (i=0; i<n_local_pgs; i++) { - int len, flag; - char *pg_str=0; - MPIDI_PG_t *pgptr; - - if (rank == root) { - if (!pg_list) { - /* FIXME: Error, the pg_list is broken */ - printf( "Unexpected end of pg_list\n" ); fflush(stdout); - break; - } - pg_str = pg_list->str; - len = pg_list->lenStr; - pg_list = pg_list->next; - } - mpi_errno = MPIR_Bcast_impl( &len, 1, MPI_INT, root, comm_p, &errflag); - if (rank != root) { - pg_str = (char *)MPL_malloc(len); - if (!pg_str) { - goto fn_exit; - } - } - mpi_errno = MPIR_Bcast_impl( pg_str, len, MPI_CHAR, root, comm_p, &errflag); - if (mpi_errno) { - if (rank != root) - MPL_free( pg_str ); - } - - if (rank != root) { - /* flag is true if the pg was created, false if it - already existed. This step - also initializes the created process group */ - MPIDI_PG_Create_from_string( pg_str, &pgptr, &flag ); - if (flag) { - /*printf( "[%d]Added pg named %s to list\n", rank, - (char *)pgptr->id ); - fflush(stdout); */ - } - MPL_free( pg_str ); - } - } - - /* Free pg_list */ - pg_list = pg_head; - - /* FIXME: We should use the PG destroy function for this, and ensure that - the PG fields are valid for that function */ - while (pg_list) { - pg_next = pg_list->next; - MPL_free( pg_list->str ); - if (pg_list->pg_id ) { - MPL_free( pg_list->pg_id ); - } - MPL_free( pg_list ); - pg_list = pg_next; - } - - fn_exit: - MPL_free(local_translation); - return mpi_errno; - fn_fail: - goto fn_exit; -} -#endif diff --git a/src/mpid/pamid/src/misc/Makefile.mk b/src/mpid/pamid/src/misc/Makefile.mk deleted file mode 100644 index 10f6587..0000000 --- a/src/mpid/pamid/src/misc/Makefile.mk +++ /dev/null @@ -1,33 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/misc/mpid_abort.c \ - src/mpid/pamid/src/misc/mpid_get_universe_size.c \ - src/mpid/pamid/src/misc/mpid_getpname.c \ - src/mpid/pamid/src/misc/mpid_mem.c \ - src/mpid/pamid/src/misc/mpid_unimpl.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/misc/mpid_abort.c b/src/mpid/pamid/src/misc/mpid_abort.c deleted file mode 100644 index 8cc669b..0000000 --- a/src/mpid/pamid/src/misc/mpid_abort.c +++ /dev/null @@ -1,103 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/misc/mpid_abort.c - * \brief Handle general operations assosicated with erroneous job termination - */ -#include <mpidimpl.h> -#include <mpix.h> - -/** - * \brief The central parts of the MPID_Abort() call - * - * \param[in] comm The communicator associated with the failure (can be null). - * \param[in] mpi_errno The MPI error associated with the failure (can be zero). - * \param[in] exit_code The requested exit code. - * \param[in] error_msg The message to display (may be NULL) - * - * This is the majority of the call to MPID_Abort(). The only - * difference is that it does not call exit. That allows it to be - * used as a test function to ensure that the output is what you would - * expect. - * - * MPIDI_Abort_core() simply uses the same params from MPID_Abort(). - */ -void MPIDI_Abort_core(MPIR_Comm * comm, int mpi_errno, int exit_code, const char *user_str) -{ - char sys_str[MPI_MAX_ERROR_STRING+5] = ""; - char comm_str[MPI_MAX_ERROR_STRING] = ""; - char world_str[MPI_MAX_ERROR_STRING] = ""; - char error_str[2*MPI_MAX_ERROR_STRING + 128]; - - if (MPIR_Process.comm_world) - { - int rank = MPIR_Process.comm_world->rank; - snprintf(world_str, sizeof(world_str), " on node %d", rank); - } - if (comm) - { - int rank = comm->rank; - int handle = comm->handle; - snprintf(comm_str, sizeof(comm_str), " (rank %d in comm %d)", rank, handle); - } - if (!user_str) - user_str = "Internal error"; - if (mpi_errno != MPI_SUCCESS) - { - char msg[MPI_MAX_ERROR_STRING] = ""; - MPIR_Err_get_string(mpi_errno, msg, MPI_MAX_ERROR_STRING, NULL); - snprintf(sys_str, sizeof(msg), " (%s)", msg); - } - - snprintf(error_str, sizeof(error_str), "Abort(%d)%s%s: %s%s\n", exit_code, world_str, comm_str, user_str, sys_str); - MPL_error_printf("%s", error_str); - - fflush(stderr); fflush(stdout); -} - -/** - * \brief The central parts of the MPID_Abort call - * \param[in] comm The communicator associated with the failure (can be null). - * \param[in] mpi_errno The MPI error associated with the failure (can be zero). - * \param[in] exit_code The requested exit code. - * \param[in] error_msg The message to display (may be NULL) - * \return MPI_ERR_INTERN - * - * This function MUST NEVER return. - */ -int MPID_Abort(MPIR_Comm * comm, int mpi_errno, int exit_code, const char *error_msg) -{ - MPIDI_Abort_core(comm, mpi_errno, exit_code, error_msg); - -#ifdef DYNAMIC_TASKING - extern int mpidi_dynamic_tasking; - if (mpidi_dynamic_tasking) - return PMI2_Abort(1,error_msg); -#endif - - /* The POE and BGQ control systems both catch the exit value for additional - * processing. If a process exits with '1' then all processes in the job - * are terminated. The requested error code is lost in this process however - * this is acceptable, but not desirable, behavior according to the MPI - * standard. - * - * On BGQ, the user may force the process (rank) that exited with '1' to core - * dump by setting the environment variable 'BG_COREDUMPONERROR=1'. - */ - exit(1); -} diff --git a/src/mpid/pamid/src/misc/mpid_get_universe_size.c b/src/mpid/pamid/src/misc/mpid_get_universe_size.c deleted file mode 100644 index 87049bb..0000000 --- a/src/mpid/pamid/src/misc/mpid_get_universe_size.c +++ /dev/null @@ -1,96 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/misc/mpid_get_universe_size.c - * \brief ??? - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include <mpidimpl.h> - -#ifdef DYNAMIC_TASKING -#ifdef USE_PMI2_API -#include "pmi2.h" -#else -#include "pmi.h" -#endif /* USE_PMI2_API */ -#endif /* DYNAMIC_TASKING */ - -extern int mpidi_dynamic_tasking; - -/* - * MPID_Get_universe_size - Get the universe size from the process manager - * - * Notes: This requires that the PMI routines are used to - * communicate with the process manager. - */ -int MPID_Get_universe_size(int * universe_size) -{ - int mpi_errno = MPI_SUCCESS; -#ifdef DYNAMIC_TASKING -#ifdef USE_PMI2_API - if(mpidi_dynamic_tasking) { - char val[PMI2_MAX_VALLEN]; - int found = 0; - char *endptr; - - mpi_errno = PMI2_Info_GetJobAttr("universeSize", val, sizeof(val), &found); - TRACE_ERR("mpi_errno from PMI2_Info_GetJobAttr=%d\n", mpi_errno); - - if (!found) { - TRACE_ERR("PMI2_Info_GetJobAttr not found\n"); - *universe_size = MPIR_UNIVERSE_SIZE_NOT_AVAILABLE; - } - else { - *universe_size = strtol(val, &endptr, 0); - TRACE_ERR("PMI2_Info_GetJobAttr found universe_size=%d\n", *universe_size); - } -#else - int pmi_errno = PMI_SUCCESS; - - pmi_errno = PMI_Get_universe_size(universe_size); - if (pmi_errno != PMI_SUCCESS) { - MPIU_ERR_SETANDJUMP1(mpi_errno, MPI_ERR_OTHER, - "**pmi_get_universe_size", - "**pmi_get_universe_size %d", pmi_errno); - } - if (*universe_size < 0) - { - *universe_size = MPIR_UNIVERSE_SIZE_NOT_AVAILABLE; - } -#endif /*USE_PMI2_API*/ - } else { - *universe_size = MPIR_UNIVERSE_SIZE_NOT_AVAILABLE; - } -#else - *universe_size = MPIR_UNIVERSE_SIZE_NOT_AVAILABLE; -#endif /*DYNAMIC_TASKING*/ - - -fn_exit: - return mpi_errno; - - /* --BEGIN ERROR HANDLING-- */ -fn_fail: - *universe_size = MPIR_UNIVERSE_SIZE_NOT_AVAILABLE; - goto fn_exit; - /* --END ERROR HANDLING-- */ -} diff --git a/src/mpid/pamid/src/misc/mpid_getpname.c b/src/mpid/pamid/src/misc/mpid_getpname.c deleted file mode 100644 index c0fbf26..0000000 --- a/src/mpid/pamid/src/misc/mpid_getpname.c +++ /dev/null @@ -1,46 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/misc/mpid_getpname.c - * \brief Device interface to MPI_Get_processor_name() - */ -#include <mpidimpl.h> - -/** - * \brief Device interface to MPI_Get_processor_name() - * \param[out] name Storage for the name as a string - * \param[in] namelen The maximum allowed length - * \param[out] resultlen The actual length written - * \returns MPI_SUCCESS - * - * All this does is convert the rank to a string and return the data - */ -int MPID_Get_processor_name(char * name, int namelen, int * resultlen) -{ - /* Get the name from PAMI */ - const char* pami_name = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_PROCESSOR_NAME).value.chararray; - /* Copy to the destination */ - strncpy(name, pami_name, namelen); - /* Ensure that there is a trailing NULL */ - if (namelen > 0) - name[namelen - 1]= '\0'; - /* Get the size of the name */ - *resultlen = strlen(name); - - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/misc/mpid_mem.c b/src/mpid/pamid/src/misc/mpid_mem.c deleted file mode 100644 index 2cd3f55..0000000 --- a/src/mpid/pamid/src/misc/mpid_mem.c +++ /dev/null @@ -1,42 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/misc/mpid_mem.c - * \brief ??? - */ -/* - * - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include <mpidimpl.h> - -void *MPID_Alloc_mem( size_t size, MPIR_Info *info_ptr ) -{ - void *ap; - ap = MPL_malloc(size); - return ap; -} - -int MPID_Free_mem( void *ptr ) -{ - int mpi_errno = MPI_SUCCESS; - MPL_free(ptr); - return mpi_errno; -} diff --git a/src/mpid/pamid/src/misc/mpid_unimpl.c b/src/mpid/pamid/src/misc/mpid_unimpl.c deleted file mode 100644 index 6b33086..0000000 --- a/src/mpid/pamid/src/misc/mpid_unimpl.c +++ /dev/null @@ -1,112 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/misc/mpid_unimpl.c - * \brief These are functions that are not supported - */ -#include <mpidimpl.h> - -#ifndef DYNAMIC_TASKING -int MPID_Close_port(const char *port_name) -{ - MPID_abort(); - return 0; -} -int MPID_Open_port(MPIR_Info *info_ptr, - char *port_name) -{ - MPID_abort(); - return 0; -} - -int MPID_Comm_accept(const char *port_name, - MPIR_Info *info_ptr, - int root, - MPIR_Comm *comm_ptr, - MPIR_Comm **newcomm) -{ - MPID_abort(); - return 0; -} -int MPID_Comm_connect(const char *port_name, - MPIR_Info *info_ptr, - int root, - MPIR_Comm *comm_ptr, - MPIR_Comm **newcomm) -{ - MPID_abort(); - return 0; -} -int MPID_Comm_disconnect(MPIR_Comm *comm_ptr) -{ - MPID_abort(); - return 0; -} -int MPID_Comm_spawn_multiple(int count, - char *array_of_commands[], - char* *array_of_argv[], - const int array_of_maxprocs[], - MPIR_Info *array_of_info[], - int root, - MPIR_Comm *comm_ptr, - MPIR_Comm **intercomm, - int array_of_errcodes[]) -{ - MPID_abort(); - return 0; -} -#endif - -int MPID_Comm_failure_ack(MPIR_Comm *comm_ptr) -{ - MPID_abort(); - return 0; -} - -int MPID_Comm_failure_get_acked(MPIR_Comm *comm_ptr, MPIR_Group **failed_group_ptr) -{ - MPID_abort(); - return 0; -} - -int MPID_Comm_get_all_failed_procs(MPIR_Comm *comm_ptr, MPIR_Group **failed_group, int tag) -{ - MPID_abort(); - return 0; -} - -int MPID_Comm_revoke(MPIR_Comm *comm_ptr, int is_remote) -{ - MPID_abort(); - return 0; -} - -int MPID_Comm_AS_enabled(MPIR_Comm *comm_ptr) -{ - /* This function must return 1 in the default case and should not be ignored - * by the implementation. */ - return 1; -} - -int MPID_Request_is_anysource(MPIR_Request *request_ptr) -{ - /* This function must not abort in the default case since it is used in many - * MPI functions. As long as the device does not implement FT, it doesn't - * matter what this function returns. */ - return 0; -} diff --git a/src/mpid/pamid/src/mpid_aint.c b/src/mpid/pamid/src/mpid_aint.c deleted file mode 100644 index e41459d..0000000 --- a/src/mpid/pamid/src/mpid_aint.c +++ /dev/null @@ -1,61 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2014 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" - -/* - Input Parameters: -+ base - base address (integer) -- disp - displacement (integer) - - Return value: - A new MPI_Aint value that is equivalent to the sum of the base and disp - arguments, where base represents a base address returned by a call - to MPI_GET_ADDRESS and disp represents a signed integer displacement. -*/ -#undef FUNCNAME -#define FUNCNAME MPID_Aint_add -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -MPI_Aint MPID_Aint_add(MPI_Aint base, MPI_Aint disp) -{ - MPI_Aint result; - - MPIR_FUNC_VERBOSE_STATE_DECL(MPID_STATE_MPID_AINT_ADD); - MPIR_FUNC_VERBOSE_ENTER(MPID_STATE_MPID_AINT_ADD); - - result = MPIR_VOID_PTR_CAST_TO_MPI_AINT ((char*)MPIR_AINT_CAST_TO_VOID_PTR(base) + disp); - - MPIR_FUNC_VERBOSE_EXIT(MPID_STATE_MPID_AINT_ADD); - return result; -} - -/* - Input Parameters: -+ addr1 - minuend address (integer) -- addr2 - subtrahend address (integer) - - Return value: - A new MPI_Aint value that is equivalent to the difference between addr1 and - addr2 arguments, where addr1 and addr2 represent addresses returned by calls - to MPI_GET_ADDRESS. -*/ -#undef FUNCNAME -#define FUNCNAME MPID_Aint_diff -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -MPI_Aint MPID_Aint_diff(MPI_Aint addr1, MPI_Aint addr2) -{ - MPI_Aint result; - - MPIR_FUNC_VERBOSE_STATE_DECL(MPID_STATE_MPID_AINT_DIFF); - MPIR_FUNC_VERBOSE_ENTER(MPID_STATE_MPID_AINT_DIFF); - - result = MPIR_PTR_DISP_CAST_TO_MPI_AINT ((char*)MPIR_AINT_CAST_TO_VOID_PTR(addr1) - (char*)MPIR_AINT_CAST_TO_VOID_PTR(addr2)); - - MPIR_FUNC_VERBOSE_EXIT(MPID_STATE_MPID_AINT_DIFF); - return result; -} diff --git a/src/mpid/pamid/src/mpid_buffer.c b/src/mpid/pamid/src/mpid_buffer.c deleted file mode 100644 index 5129d37..0000000 --- a/src/mpid/pamid/src/mpid_buffer.c +++ /dev/null @@ -1,250 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_buffer.c - * \brief MPID buffer copy - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include <mpidimpl.h> - -#ifndef MPIDI_COPY_BUFFER_SZ -#define MPIDI_COPY_BUFFER_SZ 16384 -#endif - -/** - * \brief MPID buffer copy - * - * Implements non-contiguous buffers correctly. - * - * \param[in] sbuf The address of the input buffer - * \param[in] scount The number of elements in that buffer - * \param[in] sdt The datatype of those elements - * \param[out] smpi_errno Returns errors - * \param[in] rbuf The address of the output buffer - * \param[out] rcount The number of elements in that buffer - * \param[in] rdt The datatype of those elements - * \param[out] rsz The size of the ouput data - * \param[out] rmpi_errno Returns errors - */ -void MPIDI_Buffer_copy( - const void * const sbuf, MPI_Aint scount, MPI_Datatype sdt, int * smpi_errno, - void * const rbuf, MPI_Aint rcount, MPI_Datatype rdt, intptr_t * rsz, int * rmpi_errno) -{ - int sdt_contig; - int rdt_contig; - MPI_Aint sdt_true_lb, rdt_true_lb; - intptr_t sdata_sz; - intptr_t rdata_sz; - MPIDU_Datatype* sdt_ptr; - MPIDU_Datatype* rdt_ptr; - - MPI_Aint sdt_extent; - MPI_Aint rdt_extent; - - *smpi_errno = MPI_SUCCESS; - *rmpi_errno = MPI_SUCCESS; - - /* printf("bufcopy: src count=%d dt=%d\n", scount, sdt); */ - /* printf("bufcopy: dst count=%d dt=%d\n", rcount, rdt); */ - - MPIDI_Datatype_get_info(scount, sdt, sdt_contig, sdata_sz, sdt_ptr, sdt_true_lb); - MPIDI_Datatype_get_info(rcount, rdt, rdt_contig, rdata_sz, rdt_ptr, rdt_true_lb); - - /* --BEGIN ERROR HANDLING-- */ - if (sdata_sz > rdata_sz) - { - *rmpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_RECOVERABLE, __FUNCTION__, __LINE__, MPI_ERR_TRUNCATE, "**truncate", "**truncate %d %d", sdata_sz, rdata_sz ); - sdata_sz = rdata_sz; - } - /* --END ERROR HANDLING-- */ - - if (sdata_sz == 0) - { - *rsz = 0; - goto fn_exit; - } - - if (sdt_contig && rdt_contig) - { -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on && MPIDI_cuda_is_device_buf(rbuf)) - { - cudaError_t cudaerr = CudaMemcpy(rbuf + rdt_true_lb, sbuf + sdt_true_lb, sdata_sz, cudaMemcpyHostToDevice); - } - else -#endif - memcpy((char*)rbuf + rdt_true_lb, (const char *)sbuf + sdt_true_lb, sdata_sz); - *rsz = sdata_sz; - } - else if (sdt_contig) - { -#if CUDA_AWARE_SUPPORT - // This will need to be done in two steps: - // 1 - Allocate a temp buffer which is the same size as user buffer and unpack in it. - // 2 - Copy unpacked data into user buffer from temp buffer. - if(MPIDI_Process.cuda_aware_support_on && MPIDI_cuda_is_device_buf(rbuf)) - { - MPIDU_Datatype_get_extent_macro(rdt, rdt_extent); - char *buf = MPL_malloc(rdt_extent * rcount); - memset(buf, 0, rdt_extent * rcount); - MPIDU_Segment seg; - DLOOP_Offset last; - - MPIDU_Segment_init(buf, rcount, rdt, &seg, 0); - last = sdata_sz; - MPIDU_Segment_unpack(&seg, 0, &last, (char*)sbuf + sdt_true_lb); - /* --BEGIN ERROR HANDLING-- */ - if (last != sdata_sz) - { - *rmpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_RECOVERABLE, __FUNCTION__, __LINE__, MPI_ERR_TYPE, "**dtypemismatch", 0); - } - /* --END ERROR HANDLING-- */ - - *rsz = last; - - - cudaError_t cudaerr = CudaMemcpy(rbuf + rdt_true_lb, buf, rdt_extent * rcount, cudaMemcpyHostToDevice); - - MPL_free(buf); - - goto fn_exit; - - } -#endif - - MPIDU_Segment seg; - DLOOP_Offset last; - - MPIDU_Segment_init(rbuf, rcount, rdt, &seg, 0); - last = sdata_sz; - MPIDU_Segment_unpack(&seg, 0, &last, (char*)sbuf + sdt_true_lb); - /* --BEGIN ERROR HANDLING-- */ - if (last != sdata_sz) - { - *rmpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_RECOVERABLE, __FUNCTION__, __LINE__, MPI_ERR_TYPE, "**dtypemismatch", 0); - } - /* --END ERROR HANDLING-- */ - - *rsz = last; - } - else if (rdt_contig) - { - MPIDU_Segment seg; - DLOOP_Offset last; - - MPIDU_Segment_init(sbuf, scount, sdt, &seg, 0); - last = sdata_sz; - MPIDU_Segment_pack(&seg, 0, &last, (char*)rbuf + rdt_true_lb); - /* --BEGIN ERROR HANDLING-- */ - if (last != sdata_sz) - { - *rmpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_RECOVERABLE, __FUNCTION__, __LINE__, MPI_ERR_TYPE, "**dtypemismatch", 0); - } - /* --END ERROR HANDLING-- */ - - *rsz = last; - } - else - { - char * buf; - intptr_t buf_off; - MPIDU_Segment sseg; - intptr_t sfirst; - MPIDU_Segment rseg; - intptr_t rfirst; - - buf = MPL_malloc(MPIDI_COPY_BUFFER_SZ); - /* --BEGIN ERROR HANDLING-- */ - if (buf == NULL) - { - *smpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_FATAL, __FUNCTION__, __LINE__, MPI_ERR_OTHER, "**nomem", 0); - *rmpi_errno = *smpi_errno; - *rsz = 0; - goto fn_exit; - } - /* --END ERROR HANDLING-- */ - - MPIDU_Segment_init(sbuf, scount, sdt, &sseg, 0); - MPIDU_Segment_init(rbuf, rcount, rdt, &rseg, 0); - - sfirst = 0; - rfirst = 0; - buf_off = 0; - - for(;;) - { - DLOOP_Offset last; - char * buf_end; - - if (sdata_sz - sfirst > MPIDI_COPY_BUFFER_SZ - buf_off) - { - last = sfirst + (MPIDI_COPY_BUFFER_SZ - buf_off); - } - else - { - last = sdata_sz; - } - - MPIDU_Segment_pack(&sseg, sfirst, &last, buf + buf_off); - /* --BEGIN ERROR HANDLING-- */ - MPID_assert(last > sfirst); - /* --END ERROR HANDLING-- */ - - buf_end = buf + buf_off + (last - sfirst); - sfirst = last; - - MPIDU_Segment_unpack(&rseg, rfirst, &last, buf); - /* --BEGIN ERROR HANDLING-- */ - MPID_assert(last > rfirst); - /* --END ERROR HANDLING-- */ - - rfirst = last; - - if (rfirst == sdata_sz) - { - /* successful completion */ - break; - } - - /* --BEGIN ERROR HANDLING-- */ - if (sfirst == sdata_sz) - { - /* datatype mismatch -- remaining bytes could not be unpacked */ - *rmpi_errno = MPIR_Err_create_code(MPI_SUCCESS, MPIR_ERR_RECOVERABLE, __FUNCTION__, __LINE__, MPI_ERR_TYPE, "**dtypemismatch", 0); - break; - } - /* --END ERROR HANDLING-- */ - - buf_off = sfirst - rfirst; - if (buf_off > 0) - { - memmove(buf, buf_end - buf_off, buf_off); - } - } - - *rsz = rfirst; - MPL_free(buf); - } - - fn_exit: - return; -} diff --git a/src/mpid/pamid/src/mpid_finalize.c b/src/mpid/pamid/src/mpid_finalize.c deleted file mode 100644 index 2790369..0000000 --- a/src/mpid/pamid/src/mpid_finalize.c +++ /dev/null @@ -1,149 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_finalize.c - * \brief Normal job termination code - */ -#include <mpidimpl.h> - - -#if TOKEN_FLOW_CONTROL -extern void MPIDI_close_mm(); -#endif - -#ifdef MPIDI_STATISTICS -extern pami_extension_t pe_extension; - -extern int mpidi_dynamic_tasking; -int mpidi_finalized = 0; -#ifdef DYNAMIC_TASKING -extern conn_info *_conn_info_list; -#endif - - -void MPIDI_close_pe_extension() { - extern MPIDI_printenv_t *mpich_env; - extern MPIX_stats_t *mpid_statp; - int rc; - /* PAMI_Extension_open in pami_init */ - rc = PAMI_Extension_close (pe_extension); - if (rc != PAMI_SUCCESS) { - TRACE_ERR("ERROR close PAMI_Extension failed rc %d", rc); - } - if (mpich_env) - MPL_free(mpich_env); - if (mpid_statp) - MPL_free(mpid_statp); - -} -#endif - -/** - * \brief Shut down the system - * - * At this time, no attempt is made to free memory being used for MPI structures. - * \return MPI_SUCCESS -*/ -int MPID_Finalize() -{ - pami_result_t rc; - int mpierrno = MPI_SUCCESS; - MPIR_Errflag_t errflag=MPIR_ERR_NONE; - MPIR_Barrier_impl(MPIR_Process.comm_world, &errflag); - -#ifdef MPIDI_STATISTICS - if (MPIDI_Process.mp_statistics) { - MPIDI_print_statistics(); - } - MPIDI_close_pe_extension(); -#endif - -#ifdef DYNAMIC_TASKING - mpidi_finalized = 1; - if(mpidi_dynamic_tasking) { - /* Tell the process group code that we're done with the process groups. - This will notify PMI (with PMI_Finalize) if necessary. It - also frees all PG structures, including the PG for COMM_WORLD, whose - pointer is also saved in MPIDI_Process.my_pg */ - mpierrno = MPIDI_PG_Finalize(); - if (mpierrno) { - TRACE_ERR("MPIDI_PG_Finalize returned with mpierrno=%d\n", mpierrno); - } - - MPIDI_FreeParentPort(); - } - if(_conn_info_list) - MPL_free(_conn_info_list); - MPIDI_free_all_tranid_node(); -#endif - - - /* ------------------------- */ - /* shutdown request queues */ - /* ------------------------- */ - MPIDI_Recvq_finalize(); - - PAMIX_Finalize(MPIDI_Client); - -#ifdef MPID_NEEDS_ICOMM_WORLD - MPIR_Comm_release_always(MPIR_Process.icomm_world, 0); -#endif - - MPIR_Comm_release_always(MPIR_Process.comm_self,0); - MPIR_Comm_release_always(MPIR_Process.comm_world,0); - - rc = PAMI_Context_destroyv(MPIDI_Context, MPIDI_Process.avail_contexts); - MPID_assert_always(rc == PAMI_SUCCESS); - - rc = PAMI_Client_destroy(&MPIDI_Client); - MPID_assert_always(rc == PAMI_SUCCESS); - -#ifdef MPIDI_TRACE - { int i; - for (i=0; i< MPIDI_Process.numTasks; i++) { - if (MPIDI_Trace_buf[i].R) - MPL_free(MPIDI_Trace_buf[i].R); - if (MPIDI_Trace_buf[i].PR) - MPL_free(MPIDI_Trace_buf[i].PR); - if (MPIDI_Trace_buf[i].S) - MPL_free(MPIDI_Trace_buf[i].S); - } - } - MPL_free(MPIDI_Trace_buf); -#endif - -#ifdef OUT_OF_ORDER_HANDLING - MPL_free(MPIDI_In_cntr); - MPL_free(MPIDI_Out_cntr); -#endif - - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - extern char *EagerLimit; - - if (EagerLimit) MPL_free(EagerLimit); - MPL_free(MPIDI_Token_cntr); - MPIDI_close_mm(); - #else - MPID_assert_always(0); - #endif - } - - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/mpid_improbe.c b/src/mpid/pamid/src/mpid_improbe.c deleted file mode 100644 index 97e8d7a..0000000 --- a/src/mpid/pamid/src/mpid_improbe.c +++ /dev/null @@ -1,49 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2012 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" - -int MPID_Improbe(int source, int tag, MPIR_Comm *comm, int context_offset, - int *flag, MPIR_Request **message, MPI_Status *status) -{ - const int context = comm->recvcontext_id + context_offset; - int foundp = FALSE; - MPIR_Request * rreq = NULL; - - if (source == MPI_PROC_NULL) - { - MPIR_Status_set_procnull(status); - /* We set the flag to true because an MPI_Recv with this rank will - * return immediately */ - *flag = TRUE; - *message = NULL; - return MPI_SUCCESS; - } - -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDU(source, tag, context, &foundp); -#else - int pami_source; - if(source != MPI_ANY_SOURCE) { - pami_source = MPID_VCR_GET_LPID(comm->vcr, source); - } else { - pami_source = MPI_ANY_SOURCE; - } - rreq = MPIDI_Recvq_FDU(source, pami_source, tag, context, &foundp); -#endif - if (rreq==NULL) { - MPID_Progress_poke(); - } - else { - rreq->kind = MPIR_REQUEST_KIND__MPROBE; - MPIR_Request_extract_status(rreq, status); - } - - *message = rreq; - *flag = foundp; - return MPI_SUCCESS; -} - diff --git a/src/mpid/pamid/src/mpid_imrecv.c b/src/mpid/pamid/src/mpid_imrecv.c deleted file mode 100644 index 93ad50b..0000000 --- a/src/mpid/pamid/src/mpid_imrecv.c +++ /dev/null @@ -1,91 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2012 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" - -int MPID_Imrecv(void *buf, int count, MPI_Datatype datatype, - MPIR_Request *message, MPIR_Request **rreqp) -{ - int mpi_errno = MPI_SUCCESS; - - MPIR_Request * rreq; - - /* ---------------------------------------- */ - /* NULL rank means empty request */ - /* ---------------------------------------- */ - if (unlikely(message == NULL)) - { - rreq = MPIDI_Request_create2(); - MPIR_Status_set_procnull(&rreq->status); - rreq->kind = MPIR_REQUEST_KIND__RECV; - MPIDI_Request_complete(rreq); - *rreqp = rreq; - return MPI_SUCCESS; - - } - - MPIR_Assert(message != NULL); - MPIR_Assert(message->kind == MPIR_REQUEST_KIND__MPROBE); - - /* promote the request object to be a "real" recv request */ - message->kind = MPIR_REQUEST_KIND__RECV; - - *rreqp = rreq = message; - -#ifdef OUT_OF_ORDER_HANDLING - if ((MPIDI_In_cntr[rreq->mpid.peer_pami].n_OutOfOrderMsgs>0)) - MPIDI_Recvq_process_out_of_order_msgs(rreq->mpid.peer_pami, MPIDI_Context[0]); -#endif - -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->recvs); -#endif - -#ifdef MPIDI_TRACE -{ - size_t ll; - ll = count * MPIDU_Datatype_get_basic_size(datatype); - /*MPIDI_SET_PR_REC(rreq,buf,count,ll,datatype,pami_source,rank,tag,comm,is_blocking); */ -} -#endif - - /* ----------------------------------------------------------------- */ - /* populate request with our data */ - /* We can do this because this is not a multithreaded implementation */ - /* ----------------------------------------------------------------- */ - - rreq->mpid.userbuf = buf; - rreq->mpid.userbufcount = count; - rreq->mpid.datatype = datatype; - MPIDI_RecvMsg_Unexp(rreq, buf, count, datatype); - mpi_errno = rreq->status.MPI_ERROR; - if (TOKEN_FLOW_CONTROL_ON) { - #if TOKEN_FLOW_CONTROL - if ((rreq->mpid.uebuflen) && (!(rreq->mpid.envelope.msginfo.isRzv))) { - MPIDI_Token_cntr[(rreq->mpid.peer_pami)].unmatched--; - MPIDI_Update_rettoks(rreq->mpid.peer_pami); - } - MPIDI_Must_return_tokens(MPIDI_Context[0],(rreq->mpid.peer_pami)); - #else - MPID_assert_always(0); - #endif - } - -#ifdef OUT_OF_ORDER_HANDLING - if ((MPIDI_In_cntr[rreq->mpid.peer_pami].n_OutOfOrderMsgs>0)) - MPIDI_Recvq_process_out_of_order_msgs(rreq->mpid.peer_pami, MPIDI_Context[0]); -#endif - -#ifdef MPIDI_STATISTICS - if (!(MPIR_cc_is_complete(&rreq->cc))) - { - MPID_NSTAT(mpid_statp->recvWaitsComplete); - } -#endif - - return mpi_errno; -} - diff --git a/src/mpid/pamid/src/mpid_init.c b/src/mpid/pamid/src/mpid_init.c deleted file mode 100644 index 20f3236..0000000 --- a/src/mpid/pamid/src/mpid_init.c +++ /dev/null @@ -1,1703 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_init.c - * \brief Normal job startup code - */ - -#include <stdlib.h> -#include <string.h> - -#include <mpidimpl.h> -#include "mpidi_platform.h" -#include "onesided/mpidi_onesided.h" - -#include "mpidi_util.h" - -#ifdef DYNAMIC_TASKING -#define PAMIX_CLIENT_DYNAMIC_TASKING 1032 -#define PAMIX_CLIENT_WORLD_TASKS 1033 -#define MAX_JOBID_LEN 1024 -int world_rank; -int world_size; -extern int (*mp_world_exiting_handler)(int); -extern int _mpi_world_exiting_handler(int); -#endif -int mpidi_dynamic_tasking = 0; - -#if TOKEN_FLOW_CONTROL - extern int MPIDI_mm_init(int,uint *,unsigned long *); - extern int MPIDI_tfctrl_enabled; -#endif - -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - pami_extension_t pe_extension; -#endif - -pami_client_t MPIDI_Client; -pami_context_t MPIDI_Context[MPIDI_MAX_CONTEXTS]; - -MPIDI_Process_t MPIDI_Process = { - .verbose = 0, - .statistics = 0, - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - .avail_contexts = MPIDI_MAX_CONTEXTS, - .async_progress = { - .active = 0, - .mode = ASYNC_PROGRESS_MODE_DEFAULT, - }, - .perobj = { - .context_post = { - .requested = (ASYNC_PROGRESS_MODE_DEFAULT == ASYNC_PROGRESS_MODE_LOCKED), - .active = 0, - }, - }, -#else - .avail_contexts = 1, - .async_progress = { - .active = 0, - .mode = ASYNC_PROGRESS_MODE_DISABLED, - }, - .perobj = { - .context_post = { - .requested = 0, - .active = 0, - }, - }, -#endif - .pt2pt = { - .limits = { - .application = { - .eager = { - .remote = MPIDI_EAGER_LIMIT, - .local = MPIDI_EAGER_LIMIT_LOCAL, - }, - .immediate = { - .remote = MPIDI_SHORT_LIMIT, - .local = MPIDI_SHORT_LIMIT, - }, - }, - .internal = { - .eager = { - .remote = MPIDI_EAGER_LIMIT, - .local = MPIDI_EAGER_LIMIT_LOCAL, - }, - .immediate = { - .remote = MPIDI_SHORT_LIMIT, - .local = MPIDI_SHORT_LIMIT, - }, - }, - }, - }, - .disable_internal_eager_scale = MPIDI_DISABLE_INTERNAL_EAGER_SCALE, -#if TOKEN_FLOW_CONTROL - .mp_buf_mem = BUFFER_MEM_DEFAULT, - .mp_buf_mem_max = BUFFER_MEM_DEFAULT, - .is_token_flow_control_on = 0, -#endif -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - .mp_infolevel = 0, - .mp_statistics = 0, - .mp_printenv = 0, -#endif -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - .queue_binary_search_support_on = 0, -#endif -#if CUDA_AWARE_SUPPORT - .cuda_aware_support_on = 0, -#endif - .rma_pending = 1000, - .shmem_pt2pt = 1, - .smp_detect = MPIDI_SMP_DETECT_DEFAULT, - .optimized = { - .collectives = MPIDI_OPTIMIZED_COLLECTIVE_DEFAULT, - .subcomms = 1, - .select_colls = 2, - .memory = 0, - .num_requests = 16, - }, - - .mpir_nbc = 1, - .numTasks = 0, - .typed_onesided = 0, -}; - - -struct protocol_t -{ - pami_dispatch_p2p_function func; - size_t dispatch; - size_t immediate_min; - pami_dispatch_hint_t options; -}; -static struct -{ - struct protocol_t Short; - struct protocol_t ShortSync; - struct protocol_t Eager; - struct protocol_t RVZ; - struct protocol_t Cancel; - struct protocol_t Control; - struct protocol_t WinCtrl; - struct protocol_t WinAccum; - struct protocol_t RVZ_zerobyte; - struct protocol_t WinGetAccum; - struct protocol_t WinGetAccumAck; - struct protocol_t WinAtomic; - struct protocol_t WinAtomicAck; -#ifdef DYNAMIC_TASKING - struct protocol_t Dyntask; - struct protocol_t Dyntask_disconnect; -#endif -} proto_list = { - .Short = { - .func = MPIDI_RecvShortAsyncCB, - .dispatch = MPIDI_Protocols_Short, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .ShortSync = { - .func = MPIDI_RecvShortSyncCB, - .dispatch = MPIDI_Protocols_ShortSync, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .Eager = { - .func = MPIDI_RecvCB, - .dispatch = MPIDI_Protocols_Eager, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_contiguous = PAMI_HINT_ENABLE, - .recv_copy = PAMI_HINT_ENABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .RVZ = { - .func = MPIDI_RecvRzvCB, - .dispatch = MPIDI_Protocols_RVZ, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgEnvelope), - }, - .Cancel = { - .func = MPIDI_ControlCB, - .dispatch = MPIDI_Protocols_Cancel, - .options = { - .consistency = PAMI_HINT_ENABLE, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .Control = { - .func = MPIDI_ControlCB, - .dispatch = MPIDI_Protocols_Control, - .options = { - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .WinCtrl = { - .func = MPIDI_WinControlCB, - .dispatch = MPIDI_Protocols_WinCtrl, - .options = { - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_Win_control_t), - }, - .WinAccum = { - .func = MPIDI_WinAccumCB, - .dispatch = MPIDI_Protocols_WinAccum, - .options = { - .consistency = PAMI_HINT_ENABLE, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .RVZ_zerobyte = { - .func = MPIDI_RecvRzvCB_zerobyte, - .dispatch = MPIDI_Protocols_RVZ_zerobyte, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgEnvelope), - }, - .WinGetAccum = { - .func = MPIDI_WinGetAccumCB, - .dispatch = MPIDI_Protocols_WinGetAccum, - .options = { - .consistency = PAMI_HINT_ENABLE, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_Win_GetAccMsgInfo), - }, - .WinGetAccumAck = { - .func = MPIDI_WinGetAccumAckCB, - .dispatch = MPIDI_Protocols_WinGetAccumAck, - .options = { - .consistency = PAMI_HINT_ENABLE, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_Win_GetAccMsgInfo), - }, - .WinAtomic = { - .func = MPIDI_WinAtomicCB, - .dispatch = MPIDI_Protocols_WinAtomic, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_AtomicHeader_t), - }, - .WinAtomicAck = { - .func = MPIDI_WinAtomicAckCB, - .dispatch = MPIDI_Protocols_WinAtomicAck, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_AtomicHeader_t), - }, -#ifdef DYNAMIC_TASKING - .Dyntask = { - .func = MPIDI_Recvfrom_remote_world, - .dispatch = MPIDI_Protocols_Dyntask, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, - .Dyntask_disconnect = { - .func = MPIDI_Recvfrom_remote_world_disconnect, - .dispatch = MPIDI_Protocols_Dyntask_disconnect, - .options = { - .consistency = USE_PAMI_CONSISTENCY, - .long_header = PAMI_HINT_DISABLE, - .recv_immediate = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_DISABLE, - }, - .immediate_min = sizeof(MPIDI_MsgInfo), - }, -#endif -}; - - -#undef FUNCNAME -#define FUNCNAME split_type -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -static int split_type(MPIR_Comm * comm_ptr, int stype, int key, - MPIR_Info *info_ptr, MPIR_Comm ** newcomm_ptr) -{ - MPID_Node_id_t id; - int nid; - int mpi_errno = MPI_SUCCESS; - - mpi_errno = MPID_Get_node_id(comm_ptr, comm_ptr->rank, &id); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - nid = (stype == MPI_COMM_TYPE_SHARED) ? id : MPI_UNDEFINED; - mpi_errno = MPIR_Comm_split_impl(comm_ptr, nid, key, newcomm_ptr); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - fn_exit: - return mpi_errno; - - /* --BEGIN ERROR HANDLING-- */ - fn_fail: - goto fn_exit; - /* --END ERROR HANDLING-- */ -} - -static MPIR_Commops comm_fns = { - split_type -}; - - -/* ------------------------------ */ -/* Collective selection extension */ -/* ------------------------------ */ -pami_extension_t MPIDI_Collsel_extension; -advisor_t MPIDI_Collsel_advisor; -advisor_table_t MPIDI_Collsel_advisor_table; -advisor_params_t MPIDI_Collsel_advisor_params; -char *MPIDI_Collsel_output_file; -pami_extension_collsel_advise MPIDI_Pamix_collsel_advise; -static void -MPIDI_PAMI_client_init(int* rank, int* size, int* mpidi_dynamic_tasking, char **world_tasks) -{ - /* ------------------------------------ */ - /* Initialize the MPICH->PAMI Client */ - /* ------------------------------------ */ - pami_result_t rc = PAMI_ERROR; - - pami_configuration_t config[2]; - size_t numconfigs = 0; - - /* Set the status for memory optimized collectives */ - { - char* env = getenv("PAMID_COLLECTIVES_MEMORY_OPTIMIZED"); - if (env != NULL) - MPIDI_atoi(env,&MPIDI_Process.optimized.memory); - } - -#ifdef HAVE_PAMI_CLIENT_NONCONTIG - config[0].name = PAMI_CLIENT_NONCONTIG; - if(MPIDI_Process.optimized.memory & MPIR_OPT_LVL_NONCONTIG) - config[0].value.intval = 0; // Disable non-contig, pamid doesn't use pami for non-contig data collectives so save memory - else - config[0].value.intval = 1; // Enable non-contig even though pamid doesn't use pami for non-contig data collectives, - // we still possibly want those collectives for other reasons. - ++numconfigs; -#endif -#ifdef HAVE_PAMI_CLIENT_MEMORY_OPTIMIZE - if(MPIDI_Process.optimized.memory) - { - config[numconfigs].name = PAMI_CLIENT_MEMORY_OPTIMIZE; - config[numconfigs].value.intval = MPIDI_Process.optimized.memory; - ++numconfigs; - } -#endif - - rc = PAMI_Client_create("MPI", &MPIDI_Client, config, numconfigs); - MPID_assert_always(rc == PAMI_SUCCESS); - PAMIX_Initialize(MPIDI_Client); - - - *mpidi_dynamic_tasking=0; -#ifdef DYNAMIC_TASKING - *world_tasks = NULL; - pami_result_t status = PAMI_ERROR; - - typedef pami_result_t (*dyn_task_query_fn) ( - pami_client_t client, - pami_configuration_t config[], - size_t num_configs); - dyn_task_query_fn dyn_task_query = NULL; - - pami_extension_t extension; - status = PAMI_Extension_open (MPIDI_Client, "PE_dyn_task", &extension); - if(status != PAMI_SUCCESS) - { - TRACE_ERR("Error. The PE_dyn_task extension is not implemented. result = %d\n", status); - } - - dyn_task_query = (dyn_task_query_fn) PAMI_Extension_symbol(extension, "query"); - if (dyn_task_query == (void*)NULL) { - TRACE_ERR("Err: the Dynamic Tasking extension function dyn_task_query is not implememted.\n"); - - } else { - pami_configuration_t config2[] = - { - {PAMI_CLIENT_TASK_ID, -1}, - {PAMI_CLIENT_NUM_TASKS, -1}, - {(pami_attribute_name_t)PAMIX_CLIENT_DYNAMIC_TASKING}, - {(pami_attribute_name_t)PAMIX_CLIENT_WORLD_TASKS}, - }; - - dyn_task_query(MPIDI_Client, config2, 4); - TRACE_ERR("dyn_task_query: task_id %d num_tasks %d dynamic_tasking %d world_tasks %s\n", - config2[0].value.intval, - config2[1].value.intval, - config2[2].value.intval, - config2[3].value.chararray); - *rank = world_rank = config2[0].value.intval; - *size = world_size = config2[1].value.intval; - *mpidi_dynamic_tasking = config2[2].value.intval; - *world_tasks = config2[3].value.chararray; - } - - status = PAMI_Extension_close (extension); - if(status != PAMI_SUCCESS) - { - TRACE_ERR("Error. The PE_dyn_task extension could not be closed. result = %d\n", status); - } -#endif - - if(*mpidi_dynamic_tasking == 0) { - /* ---------------------------------- */ - /* Get my rank and the process size */ - /* ---------------------------------- */ - *rank = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval; - MPIR_Process.comm_world->rank = *rank; /* Set the rank early to make tracing better */ - *size = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_NUM_TASKS).value.intval; - } - - /* --------------------------------------------------------------- */ - /* Determine if the eager point-to-point protocol for internal mpi */ - /* operations should be disabled. */ - /* --------------------------------------------------------------- */ - { - char * env = getenv("PAMID_DISABLE_INTERNAL_EAGER_TASK_LIMIT"); - if (env != NULL) - { - size_t n = strlen(env); - char * tmp = (char *) MPL_malloc(n+1); - strncpy(tmp,env,n); - if (n>0) tmp[n]=0; - - MPIDI_atoi(tmp, &MPIDI_Process.disable_internal_eager_scale); - - MPL_free(tmp); - } - - if (MPIDI_Process.disable_internal_eager_scale <= *size) - { - MPIDI_Process.pt2pt.limits.internal.eager.remote = 0; - MPIDI_Process.pt2pt.limits.internal.eager.local = 0; - MPIDI_Process.pt2pt.limits.internal.immediate.remote = 0; - MPIDI_Process.pt2pt.limits.internal.immediate.local = 0; - } - } -} - -void MPIDI_Init_collsel_extension() -{ - pami_result_t status = PAMI_ERROR; - status = PAMI_Extension_open (MPIDI_Client, "EXT_collsel", &MPIDI_Collsel_extension); - if(status == PAMI_SUCCESS) - { - if(MPIDI_Process.optimized.auto_select_colls == MPID_AUTO_SELECT_COLLS_TUNE) - { - advisor_configuration_t configuration[1]; - pami_extension_collsel_init pamix_collsel_init = - (pami_extension_collsel_init) PAMI_Extension_symbol (MPIDI_Collsel_extension, "Collsel_init"); - status = pamix_collsel_init (MPIDI_Client, configuration, 1, &MPIDI_Context[0], 1, &MPIDI_Collsel_advisor); - if(status != PAMI_SUCCESS) - { - fprintf (stderr, "Error. The collsel_init failed. result = %d\n", status); - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - } - - } - else if(MPIDI_Process.optimized.auto_select_colls == MPID_AUTO_SELECT_COLLS_ALL) - { - pami_extension_collsel_initialized pamix_collsel_initialized = - (pami_extension_collsel_initialized) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_initialized"); - if(pamix_collsel_initialized(MPIDI_Client, &MPIDI_Collsel_advisor) == 1) - { - char *collselfile; - collselfile = getenv("MP_COLLECTIVE_SELECTION_FILE"); - pami_extension_collsel_table_load pamix_collsel_table_load = - (pami_extension_collsel_table_load) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_table_load"); - if(collselfile != NULL) - status = pamix_collsel_table_load(MPIDI_Collsel_advisor, collselfile, &MPIDI_Collsel_advisor_table); - else - status = pamix_collsel_table_load(MPIDI_Collsel_advisor, "pami_tune_results.xml", &MPIDI_Collsel_advisor_table); - if (status == PAMI_SUCCESS) - { - pami_xfer_type_t *collsel_collectives = NULL; - unsigned num_collectives; - pami_extension_collsel_get_collectives pamix_collsel_get_collectives = - (pami_extension_collsel_get_collectives) PAMI_Extension_symbol(MPIDI_Collsel_extension, - "Collsel_get_collectives"); - status = pamix_collsel_get_collectives(MPIDI_Collsel_advisor_table, &collsel_collectives, &num_collectives); - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - if(collsel_collectives != NULL) - { - unsigned i = 0; - for(i = 0; i < num_collectives; i++) - { - switch(collsel_collectives[i]) - { - case PAMI_XFER_BROADCAST: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_BCAST; - break; - case PAMI_XFER_ALLREDUCE: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_ALLREDUCE; - break; - case PAMI_XFER_REDUCE: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_REDUCE; - break; - case PAMI_XFER_ALLGATHER: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_ALLGATHER; - break; - case PAMI_XFER_ALLGATHERV: -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - if(MPIDI_Process.mp_infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (collective selection) doesn't support ALLGATHERV, only ALLGATHERV_INT is supported\n"); -#endif - break; - case PAMI_XFER_ALLGATHERV_INT: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_ALLGATHERV; - break; - case PAMI_XFER_SCATTER: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_SCATTER; - break; - case PAMI_XFER_SCATTERV: -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - if(MPIDI_Process.mp_infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (collective selection) doesn't support SCATTERV, only SCATTERV_INT is supported\n"); -#endif - break; - case PAMI_XFER_SCATTERV_INT: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_SCATTERV; - break; - case PAMI_XFER_GATHER: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_GATHER; - break; - case PAMI_XFER_GATHERV: -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - if(MPIDI_Process.mp_infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (collective selection) doesn't support GATHERV, only GATHERV_INT is supported\n"); -#endif - break; - case PAMI_XFER_GATHERV_INT: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_GATHERV; - break; - case PAMI_XFER_BARRIER: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_BARRIER; - break; - case PAMI_XFER_ALLTOALL: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_ALLTOALL; - break; - case PAMI_XFER_ALLTOALLV: -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - if(MPIDI_Process.mp_infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (collective selection) doesn't support ALLTOALLV, only ALLTOALLV_INT is supported\n"); -#endif - break; - case PAMI_XFER_ALLTOALLV_INT: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_ALLTOALLV; - break; - case PAMI_XFER_SCAN: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_SCAN; - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_EXSCAN; - break; - case PAMI_XFER_REDUCE_SCATTER: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_REDUCE_SCATTER; - break; - default: - MPIDI_Process.optimized.auto_select_colls |= MPID_AUTO_SELECT_COLLS_NONE; - } - } - MPL_free(collsel_collectives); - } - } - else - { - fprintf (stderr, "Error. Collsel_table_load failed. result = %d\n", status); - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - } - } - else - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - } - else - PAMI_Extension_close(MPIDI_Collsel_extension); - } - else - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - -#ifndef __BGQ__ - //If collective selection will be disabled, check on fca and CUDA if both not required, disable pami alltogether - if(MPIDI_Process.optimized.auto_select_colls == MPID_AUTO_SELECT_COLLS_NONE && MPIDI_Process.optimized.collectives != MPID_COLL_FCA && MPIDI_Process.optimized.collectives != MPID_COLL_CUDA) - MPIDI_Process.optimized.collectives = MPID_COLL_OFF; -#endif -} - -void MPIDI_Collsel_table_generate() -{ - external_geometry_ops_t external_ops; - external_ops.geometry_create = MPIDI_Comm_create_from_pami_geom; - external_ops.geometry_destroy = MPIDI_Comm_destroy_external; - external_ops.register_algorithms = MPIDI_Register_algorithms_ext; - pami_result_t status = PAMI_SUCCESS; - pami_extension_collsel_table_generate pamix_collsel_table_generate = - (pami_extension_collsel_table_generate) PAMI_Extension_symbol (MPIDI_Collsel_extension, "Collsel_table_generate"); - - status = pamix_collsel_table_generate (MPIDI_Collsel_advisor, MPIDI_Collsel_output_file, &MPIDI_Collsel_advisor_params, &external_ops, 1); - if(status != PAMI_SUCCESS) - { - fprintf (stderr, "Error. The collsel_table_generate failed. result = %d\n", status); - } - -} - - -static void -MPIDI_PAMI_context_init(int* threading, int *size) -{ -#ifdef TRACE_ON - int requested_thread_level; - requested_thread_level = *threading; -#endif - int numTasks; - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - /* - * ASYNC_PROGRESS_MODE_LOCKED requires context post because the async thread - * will hold the context lock indefinitely; the only option for an application - * thread to interact with the context is to use context post. See discussion - * in src/mpid/pamid/src/mpid_progress.h for more information. - * - * There are three possible resolutions for the situation when context post is - * disabled and async progess mode is 'locked': - * 1. abort - * 2. silently enable context post - * 3. silently demote async progress mode to ASYNC_PROGRESS_MODE_TRIGGER - * - * For now this configuration is considered erroneous and mpi will abort. - */ - if (MPIDI_Process.async_progress.mode == ASYNC_PROGRESS_MODE_LOCKED && - MPIDI_Process.perobj.context_post.requested == 0) - MPID_Abort (NULL, 0, 1, "'locking' async progress requires context post"); - -#else /* MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ */ - /* - * ASYNC_PROGRESS_MODE_LOCKED is not applicable in the "global lock" thread - * mode. See discussion in src/mpid/pamid/src/mpid_progress.h for more - * information. - * - * This configuration is considered erroneous and mpi will abort. - */ - if (MPIDI_Process.async_progress.mode == ASYNC_PROGRESS_MODE_LOCKED) - MPID_Abort (NULL, 0, 1, "'locking' async progress not applicable"); -#endif - - /* ---------------------------------- - * Figure out the context situation - * ---------------------------------- */ -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - - /* Limit the number of requested contexts by the maximum number of contexts - * allowed. The default number of requested contexts depends on the mpich - * lock mode, 'global' or 'perobj', and may be changed before this point - * by an environment variables. - */ - if (MPIDI_Process.avail_contexts > MPIDI_MAX_CONTEXTS) - MPIDI_Process.avail_contexts = MPIDI_MAX_CONTEXTS; - - unsigned same = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_CONST_CONTEXTS).value.intval; - if (same) - { - /* Determine the maximum number of contexts supported; limit the number of - * requested contexts by this value. - */ - unsigned possible_contexts = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_NUM_CONTEXTS).value.intval; - TRACE_ERR("PAMI allows up to %u contexts; MPICH allows up to %u\n", - possible_contexts, MPIDI_Process.avail_contexts); - if (MPIDI_Process.avail_contexts > possible_contexts) - MPIDI_Process.avail_contexts = possible_contexts; - } - else - { - /* If PAMI didn't give all nodes the same number of contexts, all bets - * are off for now */ - MPIDI_Process.avail_contexts = 1; - } - - /* The number of contexts must be a power-of-two, as required by the - * MPIDI_Context_hash() function. Decrement until we hit a power-of-two */ - while(MPIDI_Process.avail_contexts & (MPIDI_Process.avail_contexts-1)) - --MPIDI_Process.avail_contexts; - MPID_assert_always(MPIDI_Process.avail_contexts); - -#else /* (MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ) */ - - /* Only a single context is supported in the 'global' mpich lock mode. - * - * A multi-context application will always perform better with the - * 'per object' mpich lock mode - regardless of whether async progress is - * enabled or not. This is because all threads, application and async - * progress, must acquire the single global lock which effectively serializes - * the threads and negates any benefit of multiple contexts. - * - * This single context limitation removes code and greatly simplifies logic. - */ - MPIDI_Process.avail_contexts = 1; - -#endif - - TRACE_ERR ("Thread-level=%d, requested=%d\n", *threading, requested_thread_level); - - MPIDI_Process.numTasks= numTasks = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_NUM_TASKS).value.intval; -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_In_cntr = MPL_calloc0(numTasks, MPIDI_In_cntr_t); - if(MPIDI_In_cntr == NULL) - MPID_abort(); - MPIDI_Out_cntr = MPL_calloc0(numTasks, MPIDI_Out_cntr_t); - if(MPIDI_Out_cntr == NULL) - MPID_abort(); - memset((void *) MPIDI_In_cntr,0, sizeof(MPIDI_In_cntr_t)); - memset((void *) MPIDI_Out_cntr,0, sizeof(MPIDI_Out_cntr_t)); -#endif - - -#ifdef MPIDI_TRACE - int i; - MPIDI_Trace_buf = MPL_calloc0(numTasks, MPIDI_Trace_buf_t); - if(MPIDI_Trace_buf == NULL) MPID_abort(); - memset((void *) MPIDI_Trace_buf,0, sizeof(MPIDI_Trace_buf_t)); - for (i=0; i < numTasks; i++) { - MPIDI_Trace_buf[i].R=MPL_calloc0(N_MSGS, recv_status); - if (MPIDI_Trace_buf[i].R==NULL) MPID_abort(); - MPIDI_Trace_buf[i].PR=MPL_calloc0(N_MSGS, posted_recv); - if (MPIDI_Trace_buf[i].PR ==NULL) MPID_abort(); - MPIDI_Trace_buf[i].S=MPL_calloc0(N_MSGS, send_status); - if (MPIDI_Trace_buf[i].S ==NULL) MPID_abort(); - } -#endif - - /* ----------------------------------- */ - /* Create the communication contexts */ - /* ----------------------------------- */ - TRACE_ERR("Creating %d contexts\n", MPIDI_Process.avail_contexts); - pami_result_t rc = PAMI_ERROR; - pami_configuration_t config[3]; - int cfgval=0; - config[cfgval].name = PAMI_CLIENT_CONST_CONTEXTS, - config[cfgval].value.intval = 1; - cfgval++; -#ifndef HAVE_ERROR_CHECKING -#ifdef OUT_OF_ORDER_HANDLING - /* disable parameter checking in PAMI - fast library only */ - config[cfgval].name = PAMI_CONTEXT_CHECK_PARAM; - config[cfgval].value.intval = 0; - cfgval++; -#endif -#endif - rc = PAMI_Context_createv(MPIDI_Client, config, cfgval, MPIDI_Context, MPIDI_Process.avail_contexts); - - MPID_assert_always(rc == PAMI_SUCCESS); - - /* --------------------------------------------- */ - /* Get collective selection advisor and cache it */ - /* --------------------------------------------- */ - /* Context is created, i.e. collective selection extension is initialized in PAMI. Now I can get the - advisor if I am not in TUNE mode. If in TUNE mode, I can init collsel and generate the table. - This is not supported on BGQ. - */ -#ifndef __BGQ_ - MPIDI_Init_collsel_extension(); -#endif - -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - MPIDI_open_pe_extension(); -#endif -} - - -static void -MPIDI_PAMI_dispath_set(size_t dispatch, - struct protocol_t * proto, - unsigned * immediate_max) -{ - size_t im_max = 0; - pami_dispatch_callback_function Recv = {.p2p = proto->func}; - MPID_assert_always(dispatch == proto->dispatch); - - if (MPIDI_Process.shmem_pt2pt == 0) - proto->options.use_shmem = PAMI_HINT_DISABLE; - - PAMIX_Dispatch_set(MPIDI_Context, - MPIDI_Process.avail_contexts, - proto->dispatch, - Recv, - proto->options, - &im_max); - TRACE_ERR("Immediate-max query: dispatch=%zu got=%zu required=%zu\n", - dispatch, im_max, proto->immediate_min); - MPID_assert_always(proto->immediate_min <= im_max); - if ((immediate_max != NULL) && (im_max < *immediate_max)) - *immediate_max = im_max; -} - - -static void -MPIDI_PAMI_dispath_init() -{ -#ifdef OUT_OF_ORDER_HANDLING - { - pami_configuration_t config; - pami_result_t rc = PAMI_ERROR; - - memset(&config, 0, sizeof(config)); - config.name = PAMI_DISPATCH_SEND_IMMEDIATE_MAX; - rc = PAMI_Dispatch_query(MPIDI_Context[0], (size_t)0, &config, 1); - if ( rc == PAMI_SUCCESS ) - { - TRACE_ERR("PAMI_DISPATCH_SEND_IMMEDIATE_MAX=%d.\n", config.value.intval, rc); - MPIDI_Process.pt2pt.limits_array[2] = config.value.intval; - } - else - { - TRACE_ERR(" Attention: PAMI_Client_query(DISPATCH_SEND_IMMEDIATE_MAX=%d) rc=%d\n", config.name, rc); - MPIDI_Process.pt2pt.limits_array[2] = 256; - } - - MPIDI_Process.pt2pt.limits_array[3] = MPIDI_Process.pt2pt.limits_array[2]; - MPIDI_Process.pt2pt.limits_array[6] = MPIDI_Process.pt2pt.limits_array[2]; - MPIDI_Process.pt2pt.limits_array[7] = MPIDI_Process.pt2pt.limits_array[2]; - } -#endif - /* ------------------------------------ */ - /* Set up the communication protocols */ - /* ------------------------------------ */ - unsigned send_immediate_max_bytes = (unsigned) -1; - MPIDI_PAMI_dispath_set(MPIDI_Protocols_Short, &proto_list.Short, &send_immediate_max_bytes); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_ShortSync, &proto_list.ShortSync, &send_immediate_max_bytes); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_Eager, &proto_list.Eager, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_RVZ, &proto_list.RVZ, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_Cancel, &proto_list.Cancel, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_Control, &proto_list.Control, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_WinCtrl, &proto_list.WinCtrl, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_WinAccum, &proto_list.WinAccum, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_RVZ_zerobyte, &proto_list.RVZ_zerobyte, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_WinGetAccum, &proto_list.WinGetAccum, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_WinGetAccumAck, &proto_list.WinGetAccumAck, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_WinAtomic, &proto_list.WinAtomic, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_WinAtomicAck, &proto_list.WinAtomicAck, NULL); - -#ifdef DYNAMIC_TASKING - MPIDI_PAMI_dispath_set(MPIDI_Protocols_Dyntask, &proto_list.Dyntask, NULL); - MPIDI_PAMI_dispath_set(MPIDI_Protocols_Dyntask_disconnect, &proto_list.Dyntask_disconnect, NULL); -#endif - - /* - * The first two protocols are our short protocols: they use - * PAMI_Send_immediate() exclusively. We get the short limit twice - * because they could be different. - * - * - The returned value is the max amount of header+data. We have - * to remove the header size. - * - * - We need to add one back, since we don't use "=" in the - * comparison. We use "if (size < short_limit) ...". - * - * - We use the min of the results just to be safe. - */ - send_immediate_max_bytes -= (sizeof(MPIDI_MsgInfo) - 1); - - if (MPIDI_Process.pt2pt.limits.application.immediate.remote > send_immediate_max_bytes) - MPIDI_Process.pt2pt.limits.application.immediate.remote = send_immediate_max_bytes; - - if (MPIDI_Process.pt2pt.limits.application.immediate.local > send_immediate_max_bytes) - MPIDI_Process.pt2pt.limits.application.immediate.local = send_immediate_max_bytes; - - if (MPIDI_Process.pt2pt.limits.internal.immediate.remote > send_immediate_max_bytes) - MPIDI_Process.pt2pt.limits.internal.immediate.remote = send_immediate_max_bytes; - - if (MPIDI_Process.pt2pt.limits.internal.immediate.local > send_immediate_max_bytes) - MPIDI_Process.pt2pt.limits.internal.immediate.local = send_immediate_max_bytes; - - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - int i; - MPIDI_mm_init(MPIDI_Process.numTasks,&MPIDI_Process.pt2pt.limits.application.eager.remote,&MPIDI_Process.mp_buf_mem); - MPIDI_Token_cntr = MPL_calloc0(MPIDI_Process.numTasks, MPIDI_Token_cntr_t); - memset((void *) MPIDI_Token_cntr,0, (sizeof(MPIDI_Token_cntr_t) * MPIDI_Process.numTasks)); - for (i=0; i < MPIDI_Process.numTasks; i++) - { - MPIDI_Token_cntr[i].tokens=MPIDI_tfctrl_enabled; - } - #else - MPID_assert_always(0); - #endif - } -} - - - -extern char **environ; -static void -printEnvVars(char *type) -{ - printf("The following %s* environment variables were specified:\n", type); - char **env; - for(env = environ; *env != 0 ; env++) - { - if(!strncasecmp(*env, type, strlen(type))) - printf(" %s\n", *env); - } -} - - -static void -MPIDI_PAMI_init(int* rank, int* size, int* threading) -{ - MPIDI_PAMI_context_init(threading, size); - - - MPIDI_PAMI_dispath_init(); - - - if ( (*rank == 0) && (MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0) ) - { - printf("MPIDI_Process.*\n" - " verbose : %u\n" - " statistics : %u\n" - " contexts : %u\n" - " async_progress : %u\n" - " context_post : %u\n" - " pt2pt.limits\n" - " application\n" - " eager\n" - " remote, local : %u, %u\n" - " short\n" - " remote, local : %u, %u\n" - " internal\n" - " eager\n" - " remote, local : %u, %u\n" - " short\n" - " remote, local : %u, %u\n" - " rma_pending : %u\n" - " shmem_pt2pt : %u\n" - " disable_internal_eager_scale : %u\n" -#if TOKEN_FLOW_CONTROL - " mp_buf_mem : %u\n" - " mp_buf_mem_max : %u\n" - " is_token_flow_control_on : %u\n" -#endif -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - " mp_infolevel : %u\n" - " mp_statistics: %u\n" - " mp_printenv : %u\n" - " mp_interrupts: %u\n" -#endif -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - " queue_binary_search_support_on : %u\n" -#endif - " optimized.collectives : %u\n" - " optimized.select_colls: %u\n" - " optimized.subcomms : %u\n" - " optimized.memory : %u\n" - " optimized.num_requests: %u\n" - " mpir_nbc : %u\n" - " numTasks : %u\n", - " typed_onesided : %u\n", - MPIDI_Process.verbose, - MPIDI_Process.statistics, - MPIDI_Process.avail_contexts, - MPIDI_Process.async_progress.mode, - MPIDI_Process.perobj.context_post.requested, - MPIDI_Process.pt2pt.limits_array[0], - MPIDI_Process.pt2pt.limits_array[1], - MPIDI_Process.pt2pt.limits_array[2], - MPIDI_Process.pt2pt.limits_array[3], - MPIDI_Process.pt2pt.limits_array[4], - MPIDI_Process.pt2pt.limits_array[5], - MPIDI_Process.pt2pt.limits_array[6], - MPIDI_Process.pt2pt.limits_array[7], - MPIDI_Process.rma_pending, - MPIDI_Process.shmem_pt2pt, - MPIDI_Process.disable_internal_eager_scale, -#if TOKEN_FLOW_CONTROL - MPIDI_Process.mp_buf_mem, - MPIDI_Process.mp_buf_mem_max, - MPIDI_Process.is_token_flow_control_on, -#endif -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - MPIDI_Process.mp_infolevel, - MPIDI_Process.mp_statistics, - MPIDI_Process.mp_printenv, - (MPIDI_Process.async_progress.mode != ASYNC_PROGRESS_MODE_DISABLED), -#endif -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - MPIDI_Process.queue_binary_search_support_on, -#endif - MPIDI_Process.optimized.collectives, - MPIDI_Process.optimized.select_colls, - MPIDI_Process.optimized.subcomms, - MPIDI_Process.optimized.memory, - MPIDI_Process.optimized.num_requests, - MPIDI_Process.mpir_nbc, - MPIDI_Process.numTasks, - MPIDI_Process.typed_onesided); - switch (*threading) - { - case MPI_THREAD_MULTIPLE: - printf("mpi thread level : 'MPI_THREAD_MULTIPLE'\n"); - break; - case MPI_THREAD_SERIALIZED: - printf("mpi thread level : 'MPI_THREAD_SERIALIZED'\n"); - break; - case MPI_THREAD_FUNNELED: - printf("mpi thread level : 'MPI_THREAD_FUNNELED'\n"); - break; - case MPI_THREAD_SINGLE: - printf("mpi thread level : 'MPI_THREAD_SINGLE'\n"); - break; - } - printf("MPICH_THREAD_GRANULARITY : '%s'\n", - (MPICH_THREAD_GRANULARITY==MPICH_THREAD_GRANULARITY__POBJ)?"per object":"global"); -#ifdef ASSERT_LEVEL - printf("ASSERT_LEVEL : %d\n", ASSERT_LEVEL); -#else - printf("ASSERT_LEVEL : not defined\n"); -#endif -#ifdef MPICH_LIBDIR - printf("MPICH_LIBDIR : %s\n", MPICH_LIBDIR); -#else - printf("MPICH_LIBDIR : not defined\n"); -#endif - printEnvVars("MPICH_"); - printEnvVars("PAMID_"); - printEnvVars("PAMI_"); - printEnvVars("COMMAGENT_"); - printEnvVars("MUSPI_"); - printEnvVars("BG_"); - } -#ifdef MPIDI_BANNER - if ((*rank == 0) && (MPIDI_Process.mp_infolevel >=1)) { - char* buf = (char *) MPL_malloc(160); - int rc = MPIDI_Banner(buf); - if ( rc == 0 ) - fprintf(stderr, "%s\n", buf); - else - TRACE_ERR("mpid_banner() return code=%d task %d",rc,*rank); - MPL_free(buf); - } -#endif -} - -#ifndef DYNAMIC_TASKING -static void -MPIDI_VCRT_init(int rank, int size) -#else -static void -MPIDI_VCRT_init(int rank, int size, char *world_tasks, MPIDI_PG_t *pg) -#endif -{ - int i, rc; - MPIR_Comm * comm; -#ifdef DYNAMIC_TASKING - int p, mpi_errno=0; - char *world_tasks_save,*cp; - char *pg_id; -#endif - - /* ------------------------------- */ - /* Initialize MPI_COMM_SELF object */ - /* ------------------------------- */ - comm = MPIR_Process.comm_self; - comm->rank = 0; - comm->remote_size = comm->local_size = 1; - rc = MPID_VCRT_Create(comm->remote_size, &comm->vcrt); - MPID_assert_always(rc == MPI_SUCCESS); - rc = MPID_VCRT_Get_ptr(comm->vcrt, &comm->vcr); - MPID_assert_always(rc == MPI_SUCCESS); - comm->vcr[0]->taskid= PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval; - -#ifdef DYNAMIC_TASKING - if(mpidi_dynamic_tasking) { - comm->vcr[0]->pg=pg->vct[rank].pg; - comm->vcr[0]->pg_rank=pg->vct[rank].pg_rank; - pg->vct[rank].taskid = comm->vcr[0]->taskid; - if(comm->vcr[0]->pg) { - TRACE_ERR("Adding ref for comm=%x vcr=%x pg=%x\n", comm, comm->vcr[0], comm->vcr[0]->pg); - MPIDI_PG_add_ref(comm->vcr[0]->pg); - } - comm->local_vcr = comm->vcr; - } - -#endif - - /* -------------------------------- */ - /* Initialize MPI_COMM_WORLD object */ - /* -------------------------------- */ - comm = MPIR_Process.comm_world; - comm->rank = rank; - comm->remote_size = comm->local_size = size; - rc = MPID_VCRT_Create(comm->remote_size, &comm->vcrt); - MPID_assert_always(rc == MPI_SUCCESS); - rc = MPID_VCRT_Get_ptr(comm->vcrt, &comm->vcr); - MPID_assert_always(rc == MPI_SUCCESS); - -#ifdef DYNAMIC_TASKING - if(mpidi_dynamic_tasking) { - i=0; - world_tasks_save = MPL_strdup(world_tasks); - if(world_tasks != NULL) { - comm->vcr[0]->taskid = atoi(strtok(world_tasks, ":")); - while( (cp=strtok(NULL, ":")) != NULL) { - comm->vcr[++i]->taskid= atoi(cp); - } - } - MPL_free(world_tasks_save); - - /* This memory will be freed by the PG_Destroy if there is an error */ - pg_id = MPL_malloc(MAX_JOBID_LEN); - - mpi_errno = PMI2_Job_GetId(pg_id, MAX_JOBID_LEN); - TRACE_ERR("PMI2_Job_GetId - pg_id=%s\n", pg_id); - - /* Initialize the connection table on COMM_WORLD from the process group's - connection table */ - for (p = 0; p < comm->local_size; p++) - { - comm->vcr[p]->pg=pg->vct[p].pg; - comm->vcr[p]->pg_rank=pg->vct[p].pg_rank; - pg->vct[p].taskid = comm->vcr[p]->taskid; - if(comm->vcr[p]->pg) { - TRACE_ERR("Adding ref for comm=%x vcr=%x pg=%x\n", comm, comm->vcr[p], comm->vcr[p]->pg); - MPIDI_PG_add_ref(comm->vcr[p]->pg); - } - /* MPID_VCR_Dup(&pg->vct[p], &(comm->vcr[p]));*/ - TRACE_ERR("comm->vcr[%d]->pg->id=%s comm->vcr[%d]->pg_rank=%d\n", p, comm->vcr[p]->pg->id, p, comm->vcr[p]->pg_rank); - TRACE_ERR("TASKID -- comm->vcr[%d]=%d\n", p, comm->vcr[p]->taskid); - } - - comm->local_vcr = comm->vcr; - }else { - for (i=0; i<size; i++) { - comm->vcr[i]->taskid = i; - TRACE_ERR("comm->vcr[%d]=%d\n", i, comm->vcr[i]->taskid); - } - TRACE_ERR("MP_I_WORLD_TASKS not SET\n"); - } -#else - for (i=0; i<size; i++) { - comm->vcr[i]->taskid = i; - TRACE_ERR("comm->vcr[%d]=%d\n", i, comm->vcr[i]->taskid); - } -#endif -} - - -/** - * \brief Initialize MPICH at ADI level. - * \param[in,out] argc Unused - * \param[in,out] argv Unused - * \param[in] requested The thread model requested by the user. - * \param[out] provided The thread model provided to user. It is the same as requested, except in VNM. - * \param[out] has_args Set to TRUE - * \param[out] has_env Set to TRUE - * \return MPI_SUCCESS - */ -int MPID_Init(int * argc, - char *** argv, - int requested, - int * provided, - int * has_args, - int * has_env) -{ - int rank, size; -#ifdef DYNAMIC_TASKING - int has_parent=0; - MPIDI_PG_t * pg=NULL; - int pg_rank=-1; - int pg_size; - int appnum,mpi_errno; - MPIR_Comm * comm; - int i,j; - pami_configuration_t config; - int world_size; -#endif - char *world_tasks; - pami_result_t rc; - - /* Override split_type */ - MPIR_Comm_fns = &comm_fns; - - /* ------------------------------------------------------------------------------- */ - /* Initialize the pami client to get the process rank; needed for env var output. */ - /* ------------------------------------------------------------------------------- */ - MPIDI_PAMI_client_init(&rank, &size, &mpidi_dynamic_tasking, &world_tasks); - TRACE_OUT("after MPIDI_PAMI_client_init rank=%d size=%d mpidi_dynamic_tasking=%d\n", rank, size, mpidi_dynamic_tasking); - - /* ------------------------------------ */ - /* Get new defaults from the Env Vars */ - /* ------------------------------------ */ - MPIDI_Env_setup(rank, requested); - - /* ----------------------------- */ - /* Initialize messager */ - /* ----------------------------- */ - if ( (MPIDI_Process.async_progress.mode == ASYNC_PROGRESS_MODE_TRIGGER) || mpidi_dynamic_tasking) - { - /* The 'trigger' async progress mode requires MPI_THREAD_MULTIPLE. - * Silently promote the thread level. - * - * See discussion in src/mpid/pamid/src/mpid_progress.h for more - * information. - */ - *provided = MPI_THREAD_MULTIPLE; - } - else - { - *provided = requested; - } -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - if (requested != MPI_THREAD_MULTIPLE) - mpich_env->single_thread=1; -#endif - MPIDI_PAMI_init(&rank, &size, provided); - -#ifdef DYNAMIC_TASKING - if (mpidi_dynamic_tasking) { - - /* - * Perform PMI initialization - */ - mpi_errno = MPIDI_InitPG( argc, argv, - has_args, has_env, &has_parent, &pg_rank, &pg ); - if (mpi_errno) { - TRACE_ERR("MPIDI_InitPG returned with mpi_errno=%d\n", mpi_errno); - } - - /* FIXME: Why are pg_size and pg_rank handled differently? */ - pg_size = MPIDI_PG_Get_size(pg); - - TRACE_ERR("MPID_Init - pg_size=%d\n", pg_size); - MPIDI_Process.my_pg = pg; /* brad : this is rework for shared memories - * because they need this set earlier - * for getting the business card - */ - MPIDI_Process.my_pg_rank = pg_rank; - - } -#endif - - /* ------------------------- */ - /* initialize request queues */ - /* ------------------------- */ - MPIDI_Recvq_init(); - - /* -------------------------------------- */ - /* Fill in some hardware structure fields */ - /* -------------------------------------- */ - extern void MPIX_Init(); - MPIX_Init(); - - /* ------------------------------- */ - /* Set process attributes */ - /* ------------------------------- */ - MPIR_Process.attrs.tag_ub = INT_MAX; - MPIR_Process.attrs.wtime_is_global = 1; - MPIR_Process.attrs.io = MPI_ANY_SOURCE; - - - /* ------------------------------- */ - /* Initialize communicator objects */ - /* ------------------------------- */ -#ifndef DYNAMIC_TASKING - MPIDI_VCRT_init(rank, size); -#else - MPIDI_VCRT_init(rank, size, world_tasks, pg); -#endif - - /* ------------------------------- */ - /* Setup optimized communicators */ - /* ------------------------------- */ - TRACE_ERR("creating world geometry\n"); - rc = PAMI_Geometry_world(MPIDI_Client, &MPIDI_Process.world_geometry); - MPID_assert_always(rc == PAMI_SUCCESS); - TRACE_ERR("calling comm_create on comm world %p\n", MPIR_Process.comm_world); - MPIR_Process.comm_world->mpid.geometry = MPIDI_Process.world_geometry; - MPIR_Process.comm_world->mpid.parent = PAMI_GEOMETRY_NULL; - MPIR_Comm_commit(MPIR_Process.comm_world); - -#ifdef DYNAMIC_TASKING - if (has_parent) { - char * parent_port; - - /* FIXME: To allow just the "root" process to - request the port and then use MPIR_Bcast_intra to - distribute it to the rest of the processes, - we need to perform the Bcast after MPI is - otherwise initialized. We could do this - by adding another MPID call that the MPI_Init(_thread) - routine would make after the rest of MPI is - initialized, but before MPI_Init returns. - In fact, such a routine could be used to - perform various checks, including parameter - consistency value (e.g., all processes have the - same environment variable values). Alternately, - we could allow a few routines to operate with - predefined parameter choices (e.g., bcast, allreduce) - for the purposes of initialization. */ - mpi_errno = MPIDI_GetParentPort(&parent_port); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_GetParentPort returned with mpi_errno=%d\n", mpi_errno); - } - - mpi_errno = MPID_Comm_connect(parent_port, NULL, 0, - MPIR_Process.comm_world, &comm); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("mpi_errno from Comm_connect=%d\n", mpi_errno); - } - - MPIR_Process.comm_parent = comm; - MPIR_Assert(MPIR_Process.comm_parent != NULL); - MPL_strncpy(comm->name, "MPI_COMM_PARENT", MPI_MAX_OBJECT_NAME); - - /* FIXME: Check that this intercommunicator gets freed in MPI_Finalize - if not already freed. */ - } - mp_world_exiting_handler = &(_mpi_world_exiting_handler); -#endif - /* ------------------------------- */ - /* Initialize timer data */ - /* ------------------------------- */ - MPIDI_PAMID_Timer_is_ready = 1; - - /* ------------------------------- */ - /* ??? */ - /* ------------------------------- */ - *has_args = TRUE; - *has_env = TRUE; -#ifdef MPIDI_PRINTENV - if (MPIDI_Process.mp_printenv) { - MPIDI_Print_mpenv(rank,size); - } -#endif - /* ----------------------------------------------- */ - /* parse params for pami_tune if in benchmark mode */ - /* ----------------------------------------------- */ - if(MPIDI_Process.optimized.auto_select_colls == MPID_AUTO_SELECT_COLLS_TUNE) - { - if(argc != NULL && argv != NULL) - { - if(MPIDI_collsel_pami_tune_parse_params(*argc, *argv) != PAMI_SUCCESS) - { - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - } - } - else - { - if(MPIDI_collsel_pami_tune_parse_params(0, NULL) != PAMI_SUCCESS) - { - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - } - } - } - return MPI_SUCCESS; -} - - -/* - * \brief This is called by MPI to let us know that MPI_Init is done. - */ -int MPID_InitCompleted() -{ - MPIDI_NBC_init(); - MPIDI_Progress_init(); - /* ----------------------------------------------- */ - /* Now all is ready.. call table generate */ - /* ----------------------------------------------- */ - if(MPIDI_Process.optimized.auto_select_colls == MPID_AUTO_SELECT_COLLS_TUNE) - { - MPIDI_Collsel_table_generate(); - MPIDI_collsel_pami_tune_cleanup(); - } - return MPI_SUCCESS; -} - -#if (MPIDI_PRINTENV || MPIDI_STATISTICS || MPIDI_BANNER) -void MPIDI_open_pe_extension() { - int rc; - /* open PE extension */ - memset(&pe_extension,0, sizeof(pami_extension_t)); - rc = PAMI_Extension_open (MPIDI_Client, "EXT_pe_extension", &pe_extension); - TRACE_ERR("PAMI_Extension_open: rc %d\n", rc); - if (rc != PAMI_SUCCESS) { - TRACE_ERR("ERROR open PAMI_Extension_open failed rc %d", rc); - MPID_assert_always(rc == PAMI_SUCCESS); - } -} - - -int MPIDI_Banner(char * bufPtr) { - char *cp, *level=NULL; - char buf[30]; - char *ASC_time; - time_t ltime; - char msgBuf[60]; - char type[64], ver_buf[64]; - struct tm *tmx,*tm1; - - /* Note: The _ibm_release_version will be expanded to a full string */ - /* ONLY IF this file is extracted from CMVC. */ - /* If this file is cloned from GIT the the string will be just */ - /* "%W%. */ - if ( strncmp(_ibm_release_version_, "%W", 2) ) { - /* IBMPE's expanded version string has release name like ppe_rbarlx */ - /* and that '_' in the name is what we are looking for. */ - /* BGQ's version string does not have a '_' in it. */ - level = strrchr(_ibm_release_version_, '_'); - if ( level ) { - level -=3; - - /* The version string generated by CMVC during weekly build has a */ - /* date which is the time when the mpidi_platform.h file is last */ - /* updated. This date can be quite old and is better removed. */ - memset(ver_buf, 0, sizeof(ver_buf)); - strncpy(ver_buf, level, sizeof(ver_buf)-1); - if ( cp = strchr(ver_buf, ',') ) *cp = '\0'; - } - } - - if(sizeof(void*) == 8) - strcpy(type, "64bit (MPI over PAMI)"); - else if(sizeof(int) == 4) - strcpy(type, "32bit (MPI over PAMI)"); - else - strcpy(type, "UNKNOWN-bit (MPI over PAMI)"); - - sprintf(msgBuf,"MPICH library was compiled on"); - - tmx=MPL_malloc(sizeof(struct tm)); - sprintf(buf,__DATE__" "__TIME__); - - if ((void *) NULL == strptime(buf, "%B %d %Y %T", tmx)) - return(1); - - /* update isdst in tmx structure */ - ltime=0; - time(<ime); - tm1 = localtime(<ime); - tmx->tm_isdst=tm1->tm_isdst; - - /* localtime updates tm_wday in tmx structure */ - ltime=mktime(tmx); - tm1 = localtime(<ime); - tmx->tm_wday = tm1->tm_wday; - ASC_time = asctime(tmx); - - if (level) { - sprintf(bufPtr, "%s %s %s %s ", type, ver_buf, msgBuf, ASC_time); - } else { - sprintf(bufPtr, "%s %s %s ", type, msgBuf, ASC_time); - - } - - MPL_free(tmx); - return MPI_SUCCESS; -} -#endif - - -static inline void -static_assertions() -{ - MPID_assert_static(sizeof(void*) == sizeof(size_t)); - MPID_assert_static(sizeof(uintptr_t) == sizeof(size_t)); -#ifdef __BGQ__ - /* MPID_VCR_GET_LPIDS relies on the VCR being a simple task list */ - MPID_VCR vcr=NULL; - MPID_assert_static(sizeof(*vcr) == sizeof(pami_task_t));/* VCR is a simple task list */ - MPID_assert_static(sizeof(vcr->taskid) == sizeof(*vcr));/* VCR is a simple task list */ - - MPID_assert_static(sizeof(MPIDI_MsgInfo) == 16); - MPID_assert_static(sizeof(uint64_t) == sizeof(size_t)); -#endif -} - -#ifdef DYNAMIC_TASKING -/* FIXME: The PG code should supply these, since it knows how the - pg_ids and other data are represented */ -int MPIDI_PG_Compare_ids(void * id1, void * id2) -{ - return (strcmp((char *) id1, (char *) id2) == 0) ? TRUE : FALSE; -} - -int MPIDI_PG_Destroy_id(MPIDI_PG_t * pg) -{ - if (pg->id != NULL) - { - TRACE_ERR("free pg id =%p pg=%p\n", pg->id, pg); - MPL_free(pg->id); - TRACE_ERR("done free pg id \n"); - } - - return MPI_SUCCESS; -} - - -int MPIDI_InitPG( int *argc, char ***argv, - int *has_args, int *has_env, int *has_parent, - int *pg_rank_p, MPIDI_PG_t **pg_p ) -{ - int pmi_errno; - int mpi_errno = MPI_SUCCESS; - int pg_rank, pg_size, appnum, pg_id_sz; - int usePMI=1; - char *pg_id; - MPIDI_PG_t *pg = 0; - - /* If we use PMI here, make the PMI calls to get the - basic values. Note that systems that return setvals == true - do not make use of PMI for the KVS routines either (it is - assumed that the discover connection information through some - other mechanism */ - /* FIXME: We may want to allow the channel to ifdef out the use - of PMI calls, or ask the channel to provide stubs that - return errors if the routines are in fact used */ - if (usePMI) { - /* - * Initialize the process manangement interface (PMI), - * and get rank and size information about our process group - */ - -#ifdef USE_PMI2_API - TRACE_ERR("Calling PMI2_Init\n"); - mpi_errno = PMI2_Init(has_parent, &pg_size, &pg_rank, &appnum); - TRACE_ERR("PMI2_Init - pg_size=%d pg_rank=%d\n", pg_size, pg_rank); - /*if (mpi_errno) MPIU_ERR_POP(mpi_errno);*/ -#else - TRACE_ERR("Calling PMI_Init\n"); - pmi_errno = PMI_Init(has_parent); - if (pmi_errno != PMI_SUCCESS) { - /* MPIU_ERR_SETANDJUMP1(mpi_errno,MPI_ERR_OTHER, "**pmi_init", - "**pmi_init %d", pmi_errno); */ - } - - pmi_errno = PMI_Get_rank(&pg_rank); - if (pmi_errno != PMI_SUCCESS) { - /*MPIU_ERR_SETANDJUMP1(mpi_errno,MPI_ERR_OTHER, "**pmi_get_rank", - "**pmi_get_rank %d", pmi_errno); */ - } - - pmi_errno = PMI_Get_size(&pg_size); - if (pmi_errno != 0) { - /*MPIU_ERR_SETANDJUMP1(mpi_errno,MPI_ERR_OTHER, "**pmi_get_size", - "**pmi_get_size %d", pmi_errno);*/ - } - - pmi_errno = PMI_Get_appnum(&appnum); - if (pmi_errno != PMI_SUCCESS) { -/* MPIU_ERR_SETANDJUMP1(mpi_errno,MPI_ERR_OTHER, "**pmi_get_appnum", - "**pmi_get_appnum %d", pmi_errno); */ - } -#endif - /* Note that if pmi is not availble, the value of MPI_APPNUM is - not set */ - if (appnum != -1) { - MPIR_Process.attrs.appnum = appnum; - } - -#ifdef USE_PMI2_API - - /* This memory will be freed by the PG_Destroy if there is an error */ - pg_id = MPL_malloc(MAX_JOBID_LEN); - - mpi_errno = PMI2_Job_GetId(pg_id, MAX_JOBID_LEN); - TRACE_ERR("PMI2_Job_GetId - pg_id=%s\n", pg_id); -#else - /* Now, initialize the process group information with PMI calls */ - /* - * Get the process group id - */ - pmi_errno = PMI_KVS_Get_name_length_max(&pg_id_sz); - if (pmi_errno != PMI_SUCCESS) { - TRACE_ERR("PMI_KVS_Get_name_length_max returned with pmi_errno=%d\n", pmi_errno); - } - - /* This memory will be freed by the PG_Destroy if there is an error */ - pg_id = MPL_malloc(pg_id_sz + 1); - - /* Note in the singleton init case, the pg_id is a dummy. - We'll want to replace this value if we join an - Process manager */ - pmi_errno = PMI_KVS_Get_my_name(pg_id, pg_id_sz); - if (pmi_errno != PMI_SUCCESS) { - TRACE_ERR("PMI_KVS_Get_my_name returned with pmi_errno=%d\n", pmi_errno); - } -#endif - } - else { - /* Create a default pg id */ - pg_id = MPL_malloc(2); - MPL_strncpy( pg_id, "0", 2 ); - } - - TRACE_ERR("pg_size=%d pg_id=%s\n", pg_size, pg_id); - /* - * Initialize the process group tracking subsystem - */ - mpi_errno = MPIDI_PG_Init(argc, argv, - MPIDI_PG_Compare_ids, MPIDI_PG_Destroy_id); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_PG_Init returned with mpi_errno=%d\n", mpi_errno); - } - - /* - * Create a new structure to track the process group for our MPI_COMM_WORLD - */ - TRACE_ERR("pg_size=%d pg_id=%p pg_id=%s\n", pg_size, pg_id, pg_id); - mpi_errno = MPIDI_PG_Create(pg_size, pg_id, &pg); - MPL_free(pg_id); - if (mpi_errno != MPI_SUCCESS) { - TRACE_ERR("MPIDI_PG_Create returned with mpi_errno=%d\n", mpi_errno); - } - - /* FIXME: We can allow the channels to tell the PG how to get - connection information by passing the pg to the channel init routine */ - if (usePMI) { - /* Tell the process group how to get connection information */ - mpi_errno = MPIDI_PG_InitConnKVS( pg ); - if (mpi_errno) - TRACE_ERR("MPIDI_PG_InitConnKVS returned with mpi_errno=%d\n", mpi_errno); - } - - /* FIXME: has_args and has_env need to come from PMI eventually... */ - *has_args = TRUE; - *has_env = TRUE; - - *pg_p = pg; - *pg_rank_p = pg_rank; - - fn_exit: - return mpi_errno; - fn_fail: - /* --BEGIN ERROR HANDLING-- */ - if (pg) { - MPIDI_PG_Destroy( pg ); - } - goto fn_exit; - /* --END ERROR HANDLING-- */ -} -#endif diff --git a/src/mpid/pamid/src/mpid_iprobe.c b/src/mpid/pamid/src/mpid_iprobe.c deleted file mode 100644 index 97e9a94..0000000 --- a/src/mpid/pamid/src/mpid_iprobe.c +++ /dev/null @@ -1,47 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_iprobe.c - * \brief ??? - */ -#include <mpidimpl.h> - - -int -MPID_Iprobe(int source, - int tag, - MPIR_Comm * comm, - int context_offset, - int *flag, - MPI_Status * status) -{ - const int context = comm->recvcontext_id + context_offset; - - if (source == MPI_PROC_NULL) - { - MPIR_Status_set_procnull(status); - /* We set the flag to true because an MPI_Recv with this rank will - * return immediately */ - *flag = TRUE; - return MPI_SUCCESS; - } - *flag = MPIDI_Recvq_FU_r(source, tag, context, status); - if (!(*flag)) - MPID_Progress_poke(); - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/mpid_mprobe.c b/src/mpid/pamid/src/mpid_mprobe.c deleted file mode 100644 index 29a27b5..0000000 --- a/src/mpid/pamid/src/mpid_mprobe.c +++ /dev/null @@ -1,41 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2012 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" - -int MPID_Mprobe(int source, int tag, MPIR_Comm *comm, int context_offset, - MPIR_Request **message, MPI_Status *status) -{ - const int context = comm->recvcontext_id + context_offset; - int foundp; - MPIR_Request * rreq = NULL; - - if (source == MPI_PROC_NULL) - { - MPIR_Status_set_procnull(status); - return MPI_SUCCESS; - } - -#ifndef OUT_OF_ORDER_HANDLING - MPID_PROGRESS_WAIT_WHILE((rreq=MPIDI_Recvq_FDU(source, tag, context, &foundp) ) == NULL ); -#else - int pami_source; - if(source != MPI_ANY_SOURCE) { - pami_source = MPID_VCR_GET_LPID(comm->vcr, source); - } else { - pami_source = MPI_ANY_SOURCE; - } - MPID_PROGRESS_WAIT_WHILE((rreq=MPIDI_Recvq_FDU(source, pami_source, tag, context, &foundp) ) == NULL ); -#endif - - if (rreq) { - rreq->kind = MPIR_REQUEST_KIND__MPROBE; - MPIR_Request_extract_status(rreq, status); - } - *message = rreq; - return MPI_SUCCESS; -} - diff --git a/src/mpid/pamid/src/mpid_mrecv.c b/src/mpid/pamid/src/mpid_mrecv.c deleted file mode 100644 index a6cde3f..0000000 --- a/src/mpid/pamid/src/mpid_mrecv.c +++ /dev/null @@ -1,34 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2012 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include "mpidimpl.h" - -int MPID_Mrecv(void *buf, int count, MPI_Datatype datatype, - MPIR_Request *message, MPI_Status *status) -{ - int mpi_errno = MPI_SUCCESS; - MPI_Request req_handle; /* dummy for MPIR_Request_complete */ - int active_flag; /* dummy for MPIR_Request_complete */ - MPIR_Request *rreq = NULL; - - if (message == NULL) { - /* treat as though MPI_MESSAGE_NO_PROC was passed */ - MPIR_Status_set_procnull(status); - return mpi_errno; - } - - /* There is no optimized MPID_Mrecv at this time because there is no real - * optimization potential in that case. MPID_Recv exists to prevent - * creating a request unnecessarily for messages that are already present - * and eligible for immediate completion. */ - mpi_errno = MPID_Imrecv(buf, count, datatype, message, &rreq); - - MPID_PROGRESS_WAIT_WHILE(!MPIR_Request_is_complete(rreq)); - - mpi_errno = MPIR_Request_complete(&req_handle, rreq, status, &active_flag); - return mpi_errno; -} - diff --git a/src/mpid/pamid/src/mpid_probe.c b/src/mpid/pamid/src/mpid_probe.c deleted file mode 100644 index 9287fec..0000000 --- a/src/mpid/pamid/src/mpid_probe.c +++ /dev/null @@ -1,41 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_probe.c - * \brief ??? - */ -#include <mpidimpl.h> - - -int -MPID_Probe(int source, - int tag, - MPIR_Comm * comm, - int context_offset, - MPI_Status * status) -{ - const int context = comm->recvcontext_id + context_offset; - - if (source == MPI_PROC_NULL) - { - MPIR_Status_set_procnull(status); - return MPI_SUCCESS; - } - MPID_PROGRESS_WAIT_WHILE(!MPIDI_Recvq_FU_r(source, tag, context, status)); - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/mpid_progress.c b/src/mpid/pamid/src/mpid_progress.c deleted file mode 100644 index fd3d321..0000000 --- a/src/mpid/pamid/src/mpid_progress.c +++ /dev/null @@ -1,355 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_progress.c - * \brief Maintain the state and rules of the MPI progress semantics - */ -#include <mpidimpl.h> - -#define MAX_PROGRESS_HOOKS 4 -typedef int (*progress_func_ptr_t) (int* made_progress); - -typedef struct progress_hook_slot { - progress_func_ptr_t func_ptr; - int active; -} progress_hook_slot_t; - -static progress_hook_slot_t progress_hooks[MAX_PROGRESS_HOOKS]; - -#undef FUNCNAME -#define FUNCNAME MPIDI_Progress_register_hook -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIDI_Progress_register_hook(int (*progress_fn)(int*), int *id) -{ - int mpi_errno = MPI_SUCCESS; - int i; - MPIR_FUNC_VERBOSE_STATE_DECL(MPID_STATE_MPIDI_PROGRESS_REGISTER_HOOK); - - MPIR_FUNC_VERBOSE_ENTER(MPID_STATE_MPIDI_PROGRESS_REGISTER_HOOK); - MPIU_THREAD_CS_ENTER(ASYNC,); - - for (i = 0; i < MAX_PROGRESS_HOOKS; i++) { - if (progress_hooks[i].func_ptr == NULL) { - progress_hooks[i].func_ptr = progress_fn; - progress_hooks[i].active = FALSE; - break; - } - } - - if (i >= MAX_PROGRESS_HOOKS) { - return MPIR_Err_create_code( MPI_SUCCESS, MPIR_ERR_RECOVERABLE, - "MPIDI_Progress_register_hook", __LINE__, - MPI_ERR_INTERN, "**progresshookstoomany", 0 ); - } - - (*id) = i; - - fn_exit: - MPIU_THREAD_CS_EXIT(ASYNC,); - MPIR_FUNC_VERBOSE_EXIT(MPID_STATE_MPIDI_PROGRESS_REGISTER_HOOK); - return mpi_errno; - - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPIDI_Progress_deregister_hook -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIDI_Progress_deregister_hook(int id) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_FUNC_VERBOSE_STATE_DECL(MPID_STATE_MPIDI_PROGRESS_DEREGISTER_HOOK); - - MPIR_FUNC_VERBOSE_ENTER(MPID_STATE_MPIDI_PROGRESS_DEREGISTER_HOOK); - MPIU_THREAD_CS_ENTER(ASYNC,); - - MPIR_Assert(id >= 0 && id < MAX_PROGRESS_HOOKS && progress_hooks[id].func_ptr != NULL); - - progress_hooks[id].func_ptr = NULL; - progress_hooks[id].active = FALSE; - - fn_exit: - MPIU_THREAD_CS_EXIT(ASYNC,); - MPIR_FUNC_VERBOSE_EXIT(MPID_STATE_MPIDI_PROGRESS_DEREGISTER_HOOK); - return mpi_errno; - - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPIDI_CH3I_Progress_activate_hook -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIDI_CH3I_Progress_activate_hook(int id) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_FUNC_VERBOSE_STATE_DECL(MPID_STATE_MPIDI_CH3I_PROGRESS_ACTIVATE_HOOK); - - MPIR_FUNC_VERBOSE_ENTER(MPID_STATE_MPIDI_CH3I_PROGRESS_ACTIVATE_HOOK); - MPIU_THREAD_CS_ENTER(MPIDCOMM,); - - MPIR_Assert(id >= 0 && id < MAX_PROGRESS_HOOKS && - progress_hooks[id].active == FALSE && progress_hooks[id].func_ptr != NULL); - progress_hooks[id].active = TRUE; - - fn_exit: - MPIU_THREAD_CS_EXIT(MPIDCOMM,); - MPIR_FUNC_VERBOSE_EXIT(MPID_STATE_MPIDI_CH3I_PROGRESS_ACTIVATE_HOOK); - return mpi_errno; - - fn_fail: - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPIDI_CH3I_Progress_deactivate_hook -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIDI_CH3I_Progress_deactivate_hook(int id) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_FUNC_VERBOSE_STATE_DECL(MPID_STATE_MPIDI_CH3I_PROGRESS_DEACTIVATE_HOOK); - - MPIR_FUNC_VERBOSE_ENTER(MPID_STATE_MPIDI_CH3I_PROGRESS_DEACTIVATE_HOOK); - MPIU_THREAD_CS_ENTER(MPIDCOMM,); - - MPIR_Assert(id >= 0 && id < MAX_PROGRESS_HOOKS && - progress_hooks[id].active == TRUE && progress_hooks[id].func_ptr != NULL); - progress_hooks[id].active = FALSE; - - fn_exit: - MPIU_THREAD_CS_EXIT(MPIDCOMM,); - MPIR_FUNC_VERBOSE_EXIT(MPID_STATE_MPIDI_CH3I_PROGRESS_DEACTIVATE_HOOK); - return mpi_errno; - - fn_fail: - goto fn_exit; -} - -void -MPIDI_Progress_init() -{ - MPIDI_Process.async_progress.active = 0; - pamix_progress_t async_progress_type=0; - - /* In the "global" mpich lock mode the only possible progress function - * is the "global mpich lock" trigger progress function. This progress - * function is ignored if async progress is disabled. - */ - pamix_progress_function progress_fn = MPIDI_Progress_async_poll; - uintptr_t i; - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - /* In the "per object" mpich lock mode the only possible progress functions - * are the "context lock" trigger progress function and the 'NULL' progress - * function. - */ - if (MPIDI_Process.async_progress.mode == ASYNC_PROGRESS_MODE_LOCKED) - progress_fn = NULL; - else - progress_fn = MPIDI_Progress_async_poll_perobj; - - MPIDI_Process.perobj.context_post.active = - MPIDI_Process.perobj.context_post.requested; -#endif - - async_progress_type = PAMIX_PROGRESS_ALL; - -#ifdef __PE__ - if (MPIDI_Process.mp_interrupts) - { - /* PAMIX_PROGRESS_TIMER, PAMIX_PROGRESS_RECV_INTERRUPT or */ - /* ASYNC_PROGRESS_ALL */ - /* mp_interrupts=0 indicates the interrupts is disabled */ - /* which is the same definition as that of PAMIX_PROGRESS_ALL. */ - /* ASYNC_PROGRESS_ALL is an internal variable which is defined */ - /* as 0x1111 and needs to be converted to corresponding pami */ - /* type PAMIX_PROGRESS_ALL */ - async_progress_type = MPIDI_Process.mp_interrupts; - if (MPIDI_Process.mp_interrupts == ASYNC_PROGRESS_ALL) - async_progress_type = PAMIX_PROGRESS_ALL; - } -#endif - - - if (MPIDI_Process.async_progress.mode != ASYNC_PROGRESS_MODE_DISABLED) - { - TRACE_ERR("Async advance beginning...\n"); - - /* Enable async progress on all contexts.*/ - for (i=0; i<MPIDI_Process.avail_contexts; ++i) - { - PAMIX_Progress_register(MPIDI_Context[i], - progress_fn, - MPIDI_Progress_async_end, - MPIDI_Progress_async_start, - (void *) i); - PAMIX_Progress_enable(MPIDI_Context[i], async_progress_type); - } - TRACE_ERR("Async advance enabled\n"); - } - - /* Initialize progress hook slots */ - for (i = 0; i < MAX_PROGRESS_HOOKS; i++) - { - progress_hooks[i].func_ptr = NULL; - progress_hooks[i].active = FALSE; - } -} - -void -MPIDI_Progress_async_start(pami_context_t context, void *cookie) -{ - /* This special async progress critical section is required for both 'global' - * and 'perobj' mpich lock modes because, regardless of the lock mode, if - * more than one context is enabled for async progress then multiple threads - * could attempt to update the shared variables at the same time. - */ - MPIU_THREAD_CS_ENTER(ASYNC,); - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - if (MPIDI_Process.async_progress.active == 0) - { - /* Asynchronous progress was completely disabled and now async progress - * is enabled for one context. Because one context is in async progress - * mode the configuration state must be updated as if all contexts were in - * async rogress mode. - */ - MPIDI_Process.perobj.context_post.active = - MPIDI_Process.perobj.context_post.requested; - } -#endif - - MPIDI_Process.async_progress.active += 1; - - MPIDI_Mutex_sync(); - - /* Required for both 'global' and 'perobj' mpich lock modes. */ - MPIU_THREAD_CS_EXIT(ASYNC,); -} - -void -MPIDI_Progress_async_end (pami_context_t context, void *cookie) -{ - /* This special async progress critical section is required for both 'global' - * and 'perobj' mpich lock modes because, regardless of the lock mode, if - * more than one context is enabled for async progress then multiple threads - * could attempt to update the shared variables at the same time. - */ - MPIU_THREAD_CS_ENTER(ASYNC,); - - MPIDI_Process.async_progress.active -= 1; - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - if (MPIDI_Process.async_progress.active == 0) - { - /* Asynchronous progress is now completely disabled on all contexts. */ - if (MPIR_ThreadInfo.thread_provided != MPI_THREAD_MULTIPLE) - { - /* The runtime is, currently, completely single threaded - there are - * no async progress threads and only one application thread. In this - * environment the context post is simply overhead and can be safely - * avoided. - */ - MPIDI_Process.perobj.context_post.active = 0; - } - } -#endif - MPIDI_Mutex_sync(); - - /* Required for both 'global' and 'perobj' mpich lock modes. */ - MPIU_THREAD_CS_EXIT(ASYNC,); -} - -void -MPIDI_Progress_async_poll (pami_context_t context, void *cookie) -{ - pami_result_t rc; - int loop_count=100; - int i, made_progress; - - /* In the "global" mpich lock mode all application threads must acquire the - * ALLFUNC global lock upon entry to the API. The async progress thread - * invoking this progress function also must first acquire the ALLFUNC - * global lock as it enters the API and before invoking advance as mpich - * callbacks invoked during advance will not take any additional locks. - */ - if (MPIU_THREAD_CS_TRY(ALLFUNC,)) /* (0==try_acquire(0)) */ - { - for (i = 0; i < MAX_PROGRESS_HOOKS; i++) { - if (progress_hooks[i].active == TRUE) { - MPIR_Assert(progress_hooks[i].func_ptr != NULL); - progress_hooks[i].func_ptr(&made_progress); - } - } - - /* There is a simplifying assertion when in the 'global' mpich lock mode - * that only a single context is supported. See the discussion in - * mpich/src/mpid/pamid/src/mpid_init.c for more information. - */ - rc = PAMI_Context_advancev (MPIDI_Context, 1, loop_count); - MPID_assert( (rc == PAMI_SUCCESS) || (rc == PAMI_EAGAIN) ); - - MPIDI_Mutex_sync(); /* is this needed ? */ - MPIU_THREAD_CS_EXIT(ALLFUNC,); /* release(0) */ -#ifdef __PE__ - if (rc == PAMI_EAGAIN) - sched_yield(); -#endif - } - - return; -} - -void -MPIDI_Progress_async_poll_perobj (pami_context_t context, void *cookie) -{ - pami_result_t rc; - int loop_count=100; - int i, made_progress; - - for (i = 0; i < MAX_PROGRESS_HOOKS; i++) { - if (progress_hooks[i].active == TRUE) { - MPIR_Assert(progress_hooks[i].func_ptr != NULL); - progress_hooks[i].func_ptr(&made_progress); - } - } - - /* In the "per object" mpich lock mode multiple application threads could be - * active within the API interacting with contexts and multiple async progress - * threads could be polling contexts in this function. The only way to force a - * serialized access to the contexts is to use the pami context locks. - */ - rc = PAMI_Context_trylock_advancev(MPIDI_Context, MPIDI_Process.avail_contexts, loop_count); - MPID_assert( (rc == PAMI_SUCCESS) || (rc == PAMI_EAGAIN) ); -#ifdef __PE__ - if (rc == PAMI_EAGAIN) - { - MPIDI_Mutex_sync(); /* is this needed ? */ - sched_yield(); - } -#endif - - return; -} - diff --git a/src/mpid/pamid/src/mpid_progress.h b/src/mpid/pamid/src/mpid_progress.h deleted file mode 100644 index be2b7fc..0000000 --- a/src/mpid/pamid/src/mpid_progress.h +++ /dev/null @@ -1,297 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_progress.h - * \brief ??? - */ - -#ifndef __src_mpid_progress_h__ -#define __src_mpid_progress_h__ - - -/** - * \defgroup MPID_PROGRESS MPID Progress engine - * - * Maintain the state and rules of the MPI progress semantics. - * - * The behavior of asynchronous progress depends on many configurable variables, - * and the requirements and/or effects of several combinations may not be - * obvious. There are certain truths and simplifying assertions that, once - * identified, can eliminate many configurations from consideration which - * greatly reduces code complexity. - * - * 1. Asynchronous progress with a NULL progress function requires context post. - * - * When a NULL progress function is specified for async progress, in some - * implementations, the context lock will be internally acquired and held - * until async progress is disabled or suspended. As such, any application - * thread that waits to acquire a context lock will block indefinitely. This - * is considered an invalid configuration. - * - * 2. Asynchronous progress with a NULL progress function requires "per object" - * mpich locks. - * - * When compiled using the "global lock" mpich thread mode the individual - * "per object" lock macros are defined as noops, yet all mpich callbacks - * that could be invoked from within a context advance only use the - * "per object" lock macros and *not* the "global" lock macros. This is - * because the async progress thread never actually leaves the API, but - * remains within the library. As designed, the mpich "global" locks are only - * acquired and released at the very top API entry and exit points. - * - * If noop "per object" lock macros are ever used the async progress threads - * will cause data corruption of mpich data structures. - * - * 3. Asynchronous progress with a non-NULL progress function using the "global" - * mpich locks requires 'MPI_THREAD_MULTIPLE'. - * - * The "global" mpich lock macros, as defined, include a check of the thread - * mode and only performs the mutex operations if the thread mode is - * 'MPI_THREAD_MULTIPLE'. When using a non-NULL progress function for async - * progress the thread that invokes the progress function is essentially - * 'outside' of the mpich library. Consequently, similar to the other API - * entry points, the "global" mpich lock must be acquired in the progress - * function before any mpich data structures may be accessed. - * - * If acquiring the "global" lock is attempted in any thread mode other - * than 'MPI_THREAD_MULTIPLE' the async progress threads will cause data - * corruption of mpich data structures. To avoid this data corruption - * problem the mpi thread mode is promoted to 'MPI_THREAD_MULTIPLE' when - * async progress with a non-NULL progress function is enabled in the - * "global" mpich lock mode. - * - * 4. Only a single context is supported in the "global" mpich lock mode. - * - * A multi-context application will always perform better with the - * "per object" mpich lock mode - regardless of whether async progress is - * enabled or not. This is because in the "global" mpich lock mode all - * threads, application and async progress, must acquire the single global - * lock which effectively serializes the threads and negates any benefit - * of multiple contexts. - * - * Asserting this single context limitation removes code, improves - * performance, and greatly simplifies logic. - * - * 5. The "global" mpich lock mode does not support context post. - * - * As the "global" mpich lock mode only supports a single context, and all - * threads, application and async progress, must first acquire the global - * lock before accessing the mpich data structures or the single context, - * it is detrimental to performance to invoke context post instead of - * directly operating on the context. - * - * Asserting this context post limitation further removes code, improves - * performance, and simplifies logic. - * - * 6. The "global" mpich lock mode does not require context lock. - * - * Access to all internal mpich data structures, including the single - * context, for all application and async progress threads, is protected by - * the "global" mpich lock upon entry to the API. This makes the context - * lock redundant. - * - * 7. The "per object" mpich lock mode assumes active asynchronous progress and - * mpi thread mode 'MPI_THREAD_MULTIPLE'. - * - * Strictly defined, the context lock is NOT required in the "per object" - * mpich lock mode if asyncronous progress is not active and the mpi thread - * level is not 'MPI_THREAD_MULTIPLE'. This means that the run environment - * is completely single threaded. - * - * An application running completely single threaded using the "per object" - * mpich lock mode will always perform worse than the same completely single - * threaded application using the "global" mpich lock mode. This is because - * a "per object" single thread will take more locks than a "global" single - * thread. - * - * Therefore, a simplifying assertion is made that, as the "per object" - * completely single threaded configuration is always worse than the "global" - * completely single threaded configuration, the "per object" mpich lock mode - * will assume the application is running a multi-threaded configuration and - * optimize the code logic accordingly. - * - * \addtogroup MPID_PROGRESS - * \{ - */ - -typedef enum -{ - ASYNC_PROGRESS_MODE_DISABLED = 0, /**< async progress is disabled */ - ASYNC_PROGRESS_MODE_LOCKED, /**< async progress uses a \c NULL progress_fn */ - ASYNC_PROGRESS_MODE_TRIGGER, /**< async progress uses a 'trigger' progress_fn */ - ASYNC_PROGRESS_MODE_COUNT /**< number of sync progress modes */ -} async_progress_mode_t; - - -/** \brief The same as MPID_Progress_wait(), since it does not block */ -#define MPID_Progress_test() MPID_Progress_wait_inline(1) -/** \brief The same as MPID_Progress_wait(), since it does not block */ -#define MPID_Progress_poke() MPID_Progress_wait_inline(1) - - -/** - * \brief A macro to easily implement advancing until a specific - * condition becomes false. - * - * \param[in] COND This is not a true parameter. It is *specifically* - * designed to be evaluated several times, allowing for the result to - * change. The condition would generally look something like - * "(cb.client == 0)". This would be used as the condition on a while - * loop. - * - * \returns MPI_SUCCESS - * - * This correctly checks the condition before attempting to loop, - * since the call to MPID_Progress_wait() may not return if the event - * is already complete. Any system *not* using this macro *must* use - * a similar check before waiting. - */ -#define MPID_PROGRESS_WAIT_WHILE(COND) \ -({ \ - while (COND) \ - MPID_Progress_wait(&__state); \ - MPI_SUCCESS; \ -}) - - -/** - * \brief A macro to easily implement advancing until a specific - * condition becomes false. - * - * \param[in] COND This is not a true parameter. It is *specifically* - * designed to be evaluated several times, allowing for the result to - * change. The condition would generally look something like - * "(cb.client == 0)". This would be used as the condition on a while - * loop. - * - * \returns MPI_SUCCESS - * - * This macro makes one pami advance regardless of the state of the COND. - */ -#define MPID_PROGRESS_WAIT_DO_WHILE(COND) \ -({ \ - do { \ - MPID_Progress_wait(&__state); \ - } while(COND); \ - MPI_SUCCESS; \ -}) - - -/** - * \brief Unused, provided since MPI calls it. - * \param[in] state The previously seen state of advance - */ -#define MPID_Progress_start(state) - -/** - * \brief Unused, provided since MPI calls it. - * \param[in] state The previously seen state of advance - */ -#define MPID_Progress_end(state) - -/** - * \brief Signal MPID_Progress_wait() that something is done/changed - * - * It is therefore important that the ADI layer include a call to - * MPIDI_Progress_signal() whenever something occurs that a node might - * be waiting on. - */ -#define MPIDI_Progress_signal() - - -#define MPID_Progress_wait(state) MPID_Progress_wait_inline(100) - - -void MPIDI_Progress_init(); -void MPIDI_Progress_async_start(pami_context_t context, void *cookie); -void MPIDI_Progress_async_end (pami_context_t context, void *cookie); -void MPIDI_Progress_async_poll (); -void MPIDI_Progress_async_poll_perobj (); - -/** - * \brief This function blocks until a request completes - * \param[in] state The previously seen state of advance - * - * It does not check what has completed, only that the counter - * changes. That happens whenever there is a call to - * MPIDI_Progress_signal(). It is therefore important that the ADI - * layer include a call to MPIDI_Progress_signal() whenever something - * occurs that a node might be waiting on. - * - */ -static inline int -MPID_Progress_wait_inline(unsigned loop_count) -{ - pami_result_t rc = 0; - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - /* - * In the "per object" thread mode the "global" lock is defined as a noop and - * therefore no locks are held at this point. - */ - if (unlikely(MPIDI_Process.async_progress.active == 0)) - { - /* - * Asynchronous progress is currently inactive; therefore this application - * thread must drive progress. - * - * It is not actually neccesary to lock the context when async progress is - * disabled and only a single context is being used and the mpi thread - * mode is not mpi thread multiple. This is unlikely to be true in the - * mpich "per obj" thread granularity compile. As an optimization, do not - * check for this condition and simply always perform the context lock. - */ - rc = PAMI_Context_trylock_advancev(MPIDI_Context, MPIDI_Process.avail_contexts, 1); - MPID_assert( (rc == PAMI_SUCCESS) || (rc == PAMI_EAGAIN) ); - } -#else - /* - * In the "global lock" thread mode the single mpich lock has already been - * acquired at this point. Any other application thread or asynchronous - * progress execution resource must also acquire this global lock and will - * block until the global lock is cycled or released. - * - * Because only one thread will enter this code at any time, including any - * async progress threads if async progress is enabled, it is unneccesary - * to acquire any context locks before the context advance operation. - * - * NOTE: The 'NULL' progress function configuation for async progress is not - * valid in the 'global' mpich lock mode. See discussion above for more - * information. - * - * NOTE: There is a simplifying assertion for the "global" mpich lock mode - * that only a single context is supported. See discussion in above for - * more information. - */ - rc = PAMI_Context_advance(MPIDI_Context[0], 1); - MPID_assert( (rc == PAMI_SUCCESS) || (rc == PAMI_EAGAIN) ); -#ifdef __PE__ - if (rc == PAMI_EAGAIN) { - MPIU_THREAD_CS_SCHED_YIELD(ALLFUNC,); /* sync, release(0), yield, acquire(0) */ - } else -#endif - MPIU_THREAD_CS_YIELD(ALLFUNC,); /* sync, release(0), acquire(0) */ -#endif - - return MPI_SUCCESS; -} - -/** \} */ - - -#endif diff --git a/src/mpid/pamid/src/mpid_recvq.c b/src/mpid/pamid/src/mpid_recvq.c deleted file mode 100644 index f0e00cb..0000000 --- a/src/mpid/pamid/src/mpid_recvq.c +++ /dev/null @@ -1,945 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_recvq.c - * \brief Functions to manage the Receive Queues - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include <mpidimpl.h> -#include "mpid_recvq.h" - -/** - * \defgroup MPID_RECVQ MPID Receive Queue management - * - * Functions to manage the Receive Queues - */ - - -/** \brief Structure to group the common recvq pointers */ -struct MPIDI_Recvq_t MPIDI_Recvq; - -#ifdef HAVE_DEBUGGER_SUPPORT -struct MPIR_Request ** const MPID_Recvq_posted_head_ptr = - &MPIDI_Recvq.posted_head; -struct MPIR_Request ** const MPID_Recvq_unexpected_head_ptr = - &MPIDI_Recvq.unexpected_head; -#endif /* HAVE_DEBUGGER_SUPPORT */ - - -/** - * \brief Set up the request queues - */ -void -MPIDI_Recvq_init() -{ - MPIDI_Recvq.posted_head = NULL; - MPIDI_Recvq.posted_tail = NULL; - MPIDI_Recvq.unexpected_head = NULL; - MPIDI_Recvq.unexpected_tail = NULL; -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - MPIDI_Recvq_init_queues(); -#endif -} - - -/** - * \brief Tear down the request queues - */ -void -MPIDI_Recvq_finalize() -{ - if(MPIDI_Process.statistics) MPIDI_Recvq_DumpQueues(MPIDI_Process.verbose); -} - - -/** - * \brief Find a request in the unexpected queue - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \return 1/0 if the matching UE request was found or not - */ -int -MPIDI_Recvq_FU(int source, int tag, int context_id, MPI_Status * status) -{ - MPIR_Request * rreq; - int found = FALSE; - void * it; -#ifdef USE_STATISTICS - unsigned search_length = 0; -#endif -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_In_cntr_t *in_cntr; - uint nMsgs=0; - pami_task_t pami_source; -#endif - - if (tag != MPI_ANY_TAG && source != MPI_ANY_SOURCE) - { -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { -#ifndef OUT_OF_ORDER_HANDLING - MPIDI_Recvq_find_in_uexp(source, tag, context_id, &rreq, &it); -#else - MPIDI_Recvq_find_in_uexp(source, tag, context_id, -1, &rreq, &it); -#endif - if (rreq != NULL) { - #ifdef OUT_OF_ORDER_HANDLING - pami_source= MPIDI_Request_getPeerRank_pami(rreq); - in_cntr=&MPIDI_In_cntr[pami_source]; - nMsgs = in_cntr->nMsgs + 1; - if( ((int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0 ) - { - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) { - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - } - #endif - found = TRUE; - if(status != MPI_STATUS_IGNORE) - *status = (rreq->status); -#ifdef OUT_OF_ORDER_HANDLING - } - else - { - MPIDI_Recvq_find_in_uexp(source, tag, context_id, nMsgs, &rreq, &it); - if (rreq != NULL) { - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - found = TRUE; - if(status != MPI_STATUS_IGNORE) - *status = (rreq->status); - } - } -#endif /* OUT_OF_ORDER_HANDLING */ - } - } - else - { -#endif /* QUEUE_BINARY_SEARCH_SUPPORT */ - rreq = MPIDI_Recvq.unexpected_head; - while (rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif - if ( (MPIDI_Request_getMatchCtxt(rreq) == context_id) && - (MPIDI_Request_getMatchRank(rreq) == source ) && - (MPIDI_Request_getMatchTag(rreq) == tag ) - ) - { - #ifdef OUT_OF_ORDER_HANDLING - pami_source= MPIDI_Request_getPeerRank_pami(rreq); - in_cntr=&MPIDI_In_cntr[pami_source]; - nMsgs = in_cntr->nMsgs + 1; - if( ((int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0 ) - { - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) { - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - } - #endif - found = TRUE; - if(status != MPI_STATUS_IGNORE) - *status = (rreq->status); - break; -#ifdef OUT_OF_ORDER_HANDLING - } -#endif - } - rreq = rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif /* QUEUE_BINARY_SEARCH_SUPPORT */ - } - else - { - MPIDI_Message_match match; - MPIDI_Message_match mask; - - match.context_id = context_id; - mask.context_id = ~0; - if (tag == MPI_ANY_TAG) - { - match.tag = 0; - mask.tag = 0; - } - else - { - match.tag = tag; - mask.tag = ~0; - } - if (source == MPI_ANY_SOURCE) - { - match.rank = 0; - mask.rank = 0; - } - else - { - match.rank = source; - mask.rank = ~0; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { -#ifndef OUT_OF_ORDER_HANDLING - MPIDI_Recvq_find_in_uexp(source, tag, match.context_id, &rreq, &it); -#else - MPIDI_Recvq_find_in_uexp(source, tag, match.context_id, -1, &rreq, &it); -#endif - if (rreq != NULL) { -#ifdef OUT_OF_ORDER_HANDLING - pami_source= MPIDI_Request_getPeerRank_pami(rreq); - in_cntr=&MPIDI_In_cntr[pami_source]; - nMsgs = in_cntr->nMsgs + 1; - if(( ( (int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0) || (source == MPI_ANY_SOURCE)) { - if(source == MPI_ANY_SOURCE) { - if((int) (nMsgs-MPIDI_Request_getMatchSeq(rreq)) < 0 ) - goto NEXT_MSG2; - } - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } -#endif - found = TRUE; - if(status != MPI_STATUS_IGNORE) - *status = (rreq->status); -#ifdef OUT_OF_ORDER_HANDLING - } - else - { - NEXT_MSG2: MPIDI_Recvq_find_in_uexp(source, tag, match.context_id, nMsgs, &rreq, &it); - if (rreq != NULL) { - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - found = TRUE; - if(status != MPI_STATUS_IGNORE) - *status = (rreq->status); - } - } -#endif /* OUT_OF_ORDER_HANDLING */ - } - } - else - { -#endif /* QUEUE_BINARY_SEARCH_SUPPORT */ - rreq = MPIDI_Recvq.unexpected_head; - while (rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif - if ( ( MPIDI_Request_getMatchCtxt(rreq) == match.context_id) && - ( (MPIDI_Request_getMatchRank(rreq) & mask.rank) == match.rank ) && - ( (MPIDI_Request_getMatchTag(rreq) & mask.tag ) == match.tag ) - ) - { -#ifdef OUT_OF_ORDER_HANDLING - pami_source= MPIDI_Request_getPeerRank_pami(rreq); - in_cntr=&MPIDI_In_cntr[pami_source]; - nMsgs = in_cntr->nMsgs + 1; - if(( ( (int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0) || (source == MPI_ANY_SOURCE)) { - if(source == MPI_ANY_SOURCE) { - if((int) (nMsgs-MPIDI_Request_getMatchSeq(rreq)) < 0 ) - goto NEXT_MSG; - } - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } -#endif - found = TRUE; - if(status != MPI_STATUS_IGNORE) - *status = (rreq->status); - break; -#ifdef OUT_OF_ORDER_HANDLING - } -#endif - } - NEXT_MSG: - rreq = rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - } - -#ifdef USE_STATISTICS - MPIDI_Statistics_time(MPIDI_Statistics.recvq.unexpected_search, search_length); -#endif - - return found; -} - - -/** - * \brief Find a request in the unexpected queue and dequeue it - * \param[in] req Find by address of request object on sender - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \return The matching UE request or NULL - */ -MPIR_Request * -MPIDI_Recvq_FDUR(MPI_Request req, int source, int tag, int context_id) -{ - MPIR_Request * prev_rreq = NULL; /* previous request in queue */ - MPIR_Request * cur_rreq = NULL; /* current request in queue */ - MPIR_Request * matching_cur_rreq = NULL; /* matching request in queue */ - MPIR_Request * matching_prev_rreq = NULL; /* previous in queue to match */ - void * it = NULL; -#ifdef USE_STATISTICS - unsigned search_length = 0; -#endif - - /* ----------------------- */ - /* first we do the finding */ - /* ----------------------- */ -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { -#ifndef OUT_OF_ORDER_HANDLING - MPIDI_Recvq_find_in_uexp(source, tag, context_id, &cur_rreq, &it); -#else - /* Passing -1 for seqno because I don't care about it here */ - MPIDI_Recvq_find_in_uexp(source, tag, context_id, -1, &cur_rreq, &it); -#endif - if (cur_rreq != NULL) { - if (MPIDI_Request_getPeerRequestH(cur_rreq) == req) - { - matching_prev_rreq = prev_rreq; - matching_cur_rreq = cur_rreq; - } - } - } - else - { -#endif - cur_rreq = MPIDI_Recvq.unexpected_head; - while (cur_rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif - if (MPIDI_Request_getPeerRequestH(cur_rreq) == req && - MPIDI_Request_getMatchCtxt(cur_rreq) == context_id && - MPIDI_Request_getMatchRank(cur_rreq) == source && - MPIDI_Request_getMatchTag(cur_rreq) == tag) - { - matching_prev_rreq = prev_rreq; - matching_cur_rreq = cur_rreq; - break; - } - prev_rreq = cur_rreq; - cur_rreq = cur_rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - - /* ----------------------- */ - /* found nothing; return */ - /* ----------------------- */ - if (matching_cur_rreq == NULL) - goto fn_exit; -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { -#ifndef OUT_OF_ORDER_HANDLING - MPIDI_Recvq_remove_uexp(source, tag, context_id, it); -#else - MPIDI_Recvq_remove_uexp(source, tag, context_id, MPIDI_Request_getMatchSeq(cur_rreq),it); -#endif - } - else -#endif - MPIDI_Recvq_remove(MPIDI_Recvq.unexpected, matching_cur_rreq, matching_prev_rreq); - - fn_exit: -#ifdef USE_STATISTICS - MPIDI_Statistics_time(MPIDI_Statistics.recvq.unexpected_search, search_length); -#endif - - return matching_cur_rreq; -} - - -/** - * \brief Out of band part of find a request in the unexpected queue and dequeue it, or allocate a new request and enqueue it in the posted queue - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \param[out] foundp TRUE iff the request was found - * \return The matching UE request or the new posted request - */ -#ifndef OUT_OF_ORDER_HANDLING -MPIR_Request * -MPIDI_Recvq_FDU(int source, int tag, int context_id, int * foundp) -#else -MPIR_Request * -MPIDI_Recvq_FDU(int source, pami_task_t pami_source, int tag, int context_id, int * foundp) -#endif -{ - int found = FALSE; - MPIR_Request * rreq = NULL; - MPIR_Request * prev_rreq; - void * it; -#ifdef USE_STATISTICS - unsigned search_length = 0; -#endif -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_In_cntr_t *in_cntr; - uint nMsgs=0; - - if(pami_source != MPI_ANY_SOURCE) { - in_cntr=&MPIDI_In_cntr[pami_source]; - nMsgs = in_cntr->nMsgs + 1; - } -#endif - - //This function is typically called when there are unexp recvs - if (tag != MPI_ANY_TAG && source != MPI_ANY_SOURCE) - { -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_Recvq_find_in_uexp(source, tag, context_id, nMsgs, &rreq, &it); -#else - MPIDI_Recvq_find_in_uexp(source, tag, context_id, &rreq, &it); -#endif - if (rreq != NULL) { -#ifdef OUT_OF_ORDER_HANDLING - if(rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) { - in_cntr->nMsgs=nMsgs; - } - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - MPIDI_Recvq_remove_uexp(source, tag, context_id, MPIDI_Request_getMatchSeq(rreq),it); -#else - MPIDI_Recvq_remove_uexp(source, tag, context_id, it); -#endif - found = TRUE; -#ifdef MPIDI_TRACE - MPIDI_Trace_buf[(rreq->mpid.partner_id)].R[(rreq->mpid.idx)].matchedInUQ2=1; -#endif - } - } - else - { -#endif - prev_rreq = NULL; - rreq = MPIDI_Recvq.unexpected_head; - while (rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif -#ifdef OUT_OF_ORDER_HANDLING - if( ((int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0 ) { -#endif - if ( (MPIDI_Request_getMatchCtxt(rreq) == context_id) && - (MPIDI_Request_getMatchRank(rreq) == source ) && - (MPIDI_Request_getMatchTag(rreq) == tag ) - ) - { -#ifdef OUT_OF_ORDER_HANDLING - if(rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) { - in_cntr->nMsgs=nMsgs; - } - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } -#endif - MPIDI_Recvq_remove(MPIDI_Recvq.unexpected, rreq, prev_rreq); - found = TRUE; - TRACE_SET_R_BIT((rreq->mpid.partner_id),(rreq->mpid.idx),fl.f.matchedInUQ2); - goto fn_exit; - } -#ifdef OUT_OF_ORDER_HANDLING - } -#endif - - prev_rreq = rreq; - rreq = rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - } - else - { - MPIDI_Message_match match; - MPIDI_Message_match mask; - - match.context_id = context_id; - mask.context_id = ~0; - if (tag == MPI_ANY_TAG) - { - match.tag = 0; - mask.tag = 0; - } - else - { - match.tag = tag; - mask.tag = ~0; - } - if (source == MPI_ANY_SOURCE) - { - match.rank = 0; - mask.rank = 0; - } - else - { - match.rank = source; - mask.rank = ~0; - } - -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { -#ifndef OUT_OF_ORDER_HANDLING - MPIDI_Recvq_find_in_uexp(source, tag, match.context_id, &rreq, &it); -#else - MPIDI_Recvq_find_in_uexp(source, tag, match.context_id, -1, &rreq, &it); -#endif - if (rreq != NULL) { -#ifdef OUT_OF_ORDER_HANDLING - if(( ( (int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0) || (source == MPI_ANY_SOURCE)) { - if(source == MPI_ANY_SOURCE) { - in_cntr = &MPIDI_In_cntr[MPIDI_Request_getPeerRank_pami(rreq)]; - nMsgs = in_cntr->nMsgs+1; - if((int) (nMsgs-MPIDI_Request_getMatchSeq(rreq)) < 0 ) - goto NEXT_MSG2; - } - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - MPIDI_Recvq_remove_uexp(MPIDI_Request_getMatchRank(rreq), MPIDI_Request_getMatchTag(rreq), match.context_id, MPIDI_Request_getMatchSeq(rreq),it); -#else /* OUT_OF_ORDER_HANDLING */ - MPIDI_Recvq_remove_uexp(MPIDI_Request_getMatchRank(rreq), MPIDI_Request_getMatchTag(rreq), match.context_id, it); -#endif/* OUT_OF_ORDER_HANDLING */ - found = TRUE; - goto fn_exit; -#ifdef OUT_OF_ORDER_HANDLING - } - NEXT_MSG2: - MPIDI_Recvq_find_in_uexp(source, tag, match.context_id, nMsgs, &rreq, &it); - if (rreq != NULL) { - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } - MPIDI_Recvq_remove_uexp(MPIDI_Request_getMatchRank(rreq), MPIDI_Request_getMatchTag(rreq), match.context_id, MPIDI_Request_getMatchSeq(rreq),it); - } -#endif /* OUT_OF_ORDER_HANDLING */ - - } - } - else - { -#endif - prev_rreq = NULL; - rreq = MPIDI_Recvq.unexpected_head; - while (rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif -#ifdef OUT_OF_ORDER_HANDLING - if(( ( (int)(nMsgs-MPIDI_Request_getMatchSeq(rreq))) >= 0) || (source == MPI_ANY_SOURCE)) { -#endif - if ( ( MPIDI_Request_getMatchCtxt(rreq) == match.context_id) && - ( (MPIDI_Request_getMatchRank(rreq) & mask.rank) == match.rank ) && - ( (MPIDI_Request_getMatchTag(rreq) & mask.tag ) == match.tag ) - ) - { -#ifdef OUT_OF_ORDER_HANDLING - if(source == MPI_ANY_SOURCE) { - in_cntr = &MPIDI_In_cntr[MPIDI_Request_getPeerRank_pami(rreq)]; - nMsgs = in_cntr->nMsgs+1; - if((int) (nMsgs-MPIDI_Request_getMatchSeq(rreq)) < 0 ) - goto NEXT_MSG; - } - if (rreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - if (MPIDI_Request_getMatchSeq(rreq) == nMsgs) - in_cntr->nMsgs=nMsgs; - MPIDI_Recvq_remove_req_from_ool(rreq,in_cntr); - } -#endif - MPIDI_Recvq_remove(MPIDI_Recvq.unexpected, rreq, prev_rreq); - found = TRUE; - goto fn_exit; - } -#ifdef OUT_OF_ORDER_HANDLING - } - NEXT_MSG: -#endif - prev_rreq = rreq; - rreq = rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - } - - fn_exit: -#ifdef USE_STATISTICS - MPIDI_Statistics_time(MPIDI_Statistics.recvq.unexpected_search, search_length); -#endif - - *foundp = found; - return rreq; -} - - -/** - * \brief Find a request in the posted queue and dequeue it - * \param[in] req Find by address of request object on sender - * \return The matching posted request or NULL - */ -int -MPIDI_Recvq_FDPR(MPIR_Request * req) -{ - MPIR_Request * cur_rreq = NULL; - MPIR_Request * prev_rreq = NULL; - int found = FALSE; -#ifdef USE_STATISTICS - unsigned search_length = 0; -#endif - -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - void * it; - if(MPIDI_Process.queue_binary_search_support_on) - { - MPIDI_Recvq_find_in_post(MPIDI_Request_getMatchRank(req), MPIDI_Request_getMatchTag(req), MPIDI_Request_getMatchCtxt(req), &cur_rreq, &it); - if (cur_rreq != NULL) { - if (cur_rreq == req) - { - MPIDI_Recvq_remove_post(MPIDI_Request_getMatchRank(req), MPIDI_Request_getMatchTag(req), MPIDI_Request_getMatchCtxt(req),it); - found = TRUE; - } - } - } - else - { -#endif /* QUEUE_BINARY_SEARCH_SUPPORT */ - cur_rreq = MPIDI_Recvq.posted_head; - - while (cur_rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif - if (cur_rreq == req) - { - MPIDI_Recvq_remove(MPIDI_Recvq.posted, cur_rreq, prev_rreq); - found = TRUE; - break; - } - - prev_rreq = cur_rreq; - cur_rreq = cur_rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif -#ifdef USE_STATISTICS - MPIDI_Statistics_time(MPIDI_Statistics.recvq.posted_search, search_length); -#endif - - return found; -} - - -/** - * \brief Find a request in the posted queue and dequeue it, or allocate a new request and enqueue it in the unexpected queue - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \param[out] foundp TRUE iff the request was found - * \return The matching posted request or the new UE request - */ -#ifndef OUT_OF_ORDER_HANDLING -MPIR_Request * -MPIDI_Recvq_FDP_or_AEU(MPIR_Request *newreq, int source, int tag, int context_id, int * foundp) -#else -MPIR_Request * -MPIDI_Recvq_FDP_or_AEU(MPIR_Request *newreq, int source, pami_task_t pami_source, int tag, int context_id, int msg_seqno, int * foundp) -#endif -{ - MPIR_Request * rreq; - int found = FALSE; - -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDP(source, tag, context_id); -#else - rreq = MPIDI_Recvq_FDP(source, pami_source, tag, context_id, msg_seqno); -#endif - - if (rreq != NULL) { - found = TRUE; - } else { -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_AEU(newreq, source, tag, context_id); -#else - rreq = MPIDI_Recvq_AEU(newreq, source, pami_source, tag, context_id, msg_seqno); -#endif - } - *foundp = found; - return rreq; -} - - -/** - * \brief Allocate a new request and enqueue it in the unexpected queue - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \return The matching posted request or the new UE request - */ -#ifndef OUT_OF_ORDER_HANDLING -MPIR_Request * -MPIDI_Recvq_AEU(MPIR_Request *newreq, int source, int tag, int context_id) -#else -MPIR_Request * -MPIDI_Recvq_AEU(MPIR_Request *newreq, int source, pami_task_t pami_source, int tag, int context_id, int msg_seqno) -#endif -{ - /* A matching request was not found in the posted queue, so we - need to allocate a new request and add it to the unexpected - queue */ - MPIR_Request *rreq; - rreq = newreq; - rreq->kind = MPIR_REQUEST_KIND__RECV; - TRACE_MEMSET_R(pami_source,msg_seqno,recv_status); - TRACE_SET_REQ_VAL(rreq->mpid.envelope.msginfo.MPIseqno,-1); - TRACE_SET_REQ_VAL(rreq->mpid.envelope.length,-1); - TRACE_SET_REQ_VAL(rreq->mpid.envelope.data,(void *) 0); -#ifndef OUT_OF_ORDER_HANDLING - MPIDI_Request_setMatch(rreq, tag, source, context_id); -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - MPIDI_Recvq_insert_uexp((void*)rreq, source, tag, context_id); - else -#endif - MPIDI_Recvq_append(MPIDI_Recvq.unexpected, rreq); -#else /* OUT_OF_ORDER_HANDLING */ - MPIR_Request *q; - MPIDI_In_cntr_t *in_cntr; - int insert, i; - - in_cntr = &MPIDI_In_cntr[pami_source]; - MPIDI_Request_setMatch(rreq, tag, source, context_id); /* mpi rank needed */ - MPIDI_Request_setPeerRank_pami(rreq, pami_source); - MPIDI_Request_setPeerRank_comm(rreq, source); - MPIDI_Request_setMatchSeq(rreq, msg_seqno); -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { - MPIDI_Recvq_insert_uexp((void*)rreq, source, tag, context_id, msg_seqno); - } - else - { -#endif - if (!in_cntr->n_OutOfOrderMsgs) { - MPIDI_Recvq_append(MPIDI_Recvq.unexpected, rreq); - } else { - q=in_cntr->OutOfOrderList; - insert=0; - for (i=1; i<=in_cntr->n_OutOfOrderMsgs; i++) { - if ( context_id == MPIDI_Request_getMatchCtxt(q)) { - if (((int)(msg_seqno - MPIDI_Request_getMatchSeq(q))) < 0) { - MPIDI_Recvq_insert(MPIDI_Recvq.unexpected, q, rreq); - insert=1; - break; - } - } - q=q->mpid.nextR; - } - if (!insert) { - MPIDI_Recvq_append(MPIDI_Recvq.unexpected, rreq); - } - } - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),req,rreq); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),msgid,msg_seqno); - TRACE_SET_R_BIT(pami_source,(msg_seqno & SEQMASK),fl.f.ool); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),rtag,tag); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),rctx,context_id); - TRACE_SET_REQ_VAL(rreq->mpid.idx,(msg_seqno & SEQMASK)); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),rsource,pami_source); - TRACE_SET_REQ_VAL(rreq->mpid.partner_id,pami_source); -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - - if (((int)(in_cntr->nMsgs - msg_seqno)) < 0) { /* seqno > nMsgs, out of order */ - MPIDI_Recvq_enqueue_ool(pami_source,rreq); - } -#endif/* OUT_OF_ORDER_HANDLING */ - - return rreq; -} - - -/** - * \brief Dump the queues - */ -void -MPIDI_Recvq_DumpQueues(int verbose) -{ - if(verbose < MPIDI_VERBOSE_SUMMARY_ALL) - return; - - MPIR_Request * rreq = MPIDI_Recvq.posted_head; - unsigned i=0, numposted=0, numue=0; - unsigned postedbytes=0, uebytes=0; - - if(verbose >= MPIDI_VERBOSE_DETAILS_ALL) - fprintf(stderr,"Posted Queue:\n-------------\n"); - while (rreq != NULL) { - if(verbose >= MPIDI_VERBOSE_DETAILS_ALL) - fprintf(stderr, "P %d: MPItag=%d MPIrank=%d ctxt=%d count=%d\n", - i++, - MPIDI_Request_getMatchTag(rreq), - MPIDI_Request_getMatchRank(rreq), - MPIDI_Request_getMatchCtxt(rreq), - rreq->mpid.userbufcount - ); - numposted++; - postedbytes+=rreq->mpid.userbufcount; - rreq = rreq->mpid.next; - } - fprintf(stderr, "Posted Requests %d, Total Mem: %d bytes\n", - numposted, postedbytes); - - - i=0; - rreq = MPIDI_Recvq.unexpected_head; - if(verbose >= MPIDI_VERBOSE_DETAILS_ALL) - fprintf(stderr, "Unexpected Queue:\n-----------------\n"); - while (rreq != NULL) { - if(verbose >= MPIDI_VERBOSE_DETAILS_ALL) -#ifndef OUT_OF_ORDER_HANDLING - fprintf(stderr, "UE %d: MPItag=%d MPIrank=%d ctxt=%d uebuf=%p uebuflen=%u\n", - i++, - MPIDI_Request_getMatchTag(rreq), - MPIDI_Request_getMatchRank(rreq), - MPIDI_Request_getMatchCtxt(rreq), - rreq->mpid.uebuf, - rreq->mpid.uebuflen); -#else - fprintf(stderr, "UE %d: MPItag=%d MPIrank=%d pami_task_id=%d MPIseq=%d ctxt=%d uebuf=%p uebuflen=%u\n", - i++, - MPIDI_Request_getMatchTag(rreq), - MPIDI_Request_getMatchRank(rreq), - MPIDI_Request_getPeerRank_pami(rreq), - MPIDI_Request_getMatchSeq(rreq), - MPIDI_Request_getMatchCtxt(rreq), - rreq->mpid.uebuf, - rreq->mpid.uebuflen); -#endif - numue++; - uebytes+=rreq->mpid.uebuflen; - rreq = rreq->mpid.next; - } - fprintf(stderr, "Unexpected Requests %d, Total Mem: %d bytes\n", - numue, uebytes); -} - - -#ifdef OUT_OF_ORDER_HANDLING -/** - * Insert a request in the OutOfOrderList, make sure this list is - * arranged in the ascending order. - */ -void MPIDI_Recvq_enqueue_ool(pami_task_t src, MPIR_Request *req) -{ - MPIR_Request *q; - void *head; - int insert,i; - MPIDI_In_cntr_t *in_cntr; - - in_cntr=&MPIDI_In_cntr[src]; - if (in_cntr->n_OutOfOrderMsgs != 0) { - head=in_cntr->OutOfOrderList; - q=in_cntr->OutOfOrderList; - insert=0; - MPID_assert(q->mpid.nextR != NULL); - while(q->mpid.nextR != head) { - if (((int)(MPIDI_Request_getMatchSeq(q) - MPIDI_Request_getMatchSeq(req))) > 0) { - insert=1; - break; - } - q=q->mpid.nextR; - } - if (insert) { - MPIDI_Recvq_insert_ool(q,req); - if (q == head) { /* 1st element in the list */ - in_cntr->OutOfOrderList=req; - } - } else { - if (((int)(MPIDI_Request_getMatchSeq(q) - MPIDI_Request_getMatchSeq(req))) > 0) { - MPIDI_Recvq_insert_ool(q,req); - if (q == head) { /* 1st element in the list */ - in_cntr->OutOfOrderList=req; - } - } else { - MPIDI_Recvq_insert_ool((MPIR_Request *)q->mpid.nextR,req); - } - } - } else { /* empty list */ - in_cntr->OutOfOrderList=req; - req->mpid.prevR=req; - req->mpid.nextR=req; - } - in_cntr->n_OutOfOrderMsgs++; -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->unorderedMsgs); -#endif -} /* void MPIDI_Recvq_insert_ool(pami_task_t src, MPIR_Request *N) */ - - -/** - * MPIDI_Recvq_insert_ool: place e between q and q->prevR - * - */ -void MPIDI_Recvq_insert_ool(MPIR_Request *q,MPIR_Request *e) -{ - (e)->mpid.prevR = (q)->mpid.prevR; - ((MPIR_Request *)((q)->mpid.prevR))->mpid.nextR = (e); - (e)->mpid.nextR = (q); - (q)->mpid.prevR = (e); -} -#endif diff --git a/src/mpid/pamid/src/mpid_recvq.h b/src/mpid/pamid/src/mpid_recvq.h deleted file mode 100644 index 12b8733..0000000 --- a/src/mpid/pamid/src/mpid_recvq.h +++ /dev/null @@ -1,425 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_recvq.h - * \brief Accessors and actors for MPID Requests - */ -#ifndef __src_pt2pt_mpidi_recvq_h__ -#define __src_pt2pt_mpidi_recvq_h__ - - -struct MPIDI_Recvq_t -{ - struct MPIR_Request * posted_head; /**< \brief The Head of the Posted queue */ - struct MPIR_Request * posted_tail; /**< \brief The Tail of the Posted queue */ - struct MPIR_Request * unexpected_head; /**< \brief The Head of the Unexpected queue */ - struct MPIR_Request * unexpected_tail; /**< \brief The Tail of the Unexpected queue */ -}; -extern struct MPIDI_Recvq_t MPIDI_Recvq; - - -#ifndef OUT_OF_ORDER_HANDLING -#define MPIDI_Recvq_append(__Q, __req) \ -({ \ - /* ---------------------------------------------- */ \ - /* The tail request should point to the new one */ \ - /* ---------------------------------------------- */ \ - if (__Q ## _tail != NULL) \ - __Q ## _tail->mpid.next = __req; \ - else \ - __Q ## _head = __req; \ - /* ------------------------------------------ */ \ - /* The tail should point to the new request */ \ - /* ------------------------------------------ */ \ - __Q ## _tail = __req; \ -}) -#else -#define MPIDI_Recvq_append(__Q, __req) \ -({ \ - /* ---------------------------------------------- */ \ - /* The tail request should point to the new one */ \ - /* ---------------------------------------------- */ \ - if (__Q ## _tail != NULL) { \ - __Q ## _tail->mpid.next = __req; \ - __req->mpid.prev = __Q ## _tail; \ - } \ - else { \ - __Q ## _head = __req; \ - __req->mpid.prev = NULL; \ - } \ - /* ------------------------------------------ */ \ - /* The tail should point to the new request */ \ - /* ------------------------------------------ */ \ - __Q ## _tail = __req; \ -}) -#endif - - -#ifndef OUT_OF_ORDER_HANDLING -#define MPIDI_Recvq_remove(__Q, __req, __prev) \ -({ \ - /* --------------------------------------------- */ \ - /* Patch the next pointers to skip the request */ \ - /* --------------------------------------------- */ \ - if (__prev != NULL) \ - __prev->mpid.next = __req->mpid.next; \ - else \ - __Q ## _head = __req->mpid.next; \ - /* ------------------------------------------- */ \ - /* Set tail pointer if removing the last one */ \ - /* ------------------------------------------- */ \ - if (__req->mpid.next == NULL) \ - __Q ## _tail = __prev; \ -}) -#else -#define MPIDI_Recvq_remove(__Q, __req, __prev) \ -({ \ - /* --------------------------------------------- */ \ - /* Patch the next pointers to skip the request */ \ - /* --------------------------------------------- */ \ - if (__prev != NULL) { \ - __prev->mpid.next = __req->mpid.next; \ - } \ - else \ - __Q ## _head = __req->mpid.next; \ - /* ------------------------------------------- */ \ - /* Set tail pointer if removing the last one */ \ - /* ------------------------------------------- */ \ - if (__req->mpid.next == NULL) \ - __Q ## _tail = __prev; \ - else \ - (__req->mpid.next)->mpid.prev = __prev; \ -}) -#endif - - -/** - * \brief A thread-safe version of MPIDI_Recvq_FU - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \return 1/0 if the matching UE request was found or not - */ -static inline int -MPIDI_Recvq_FU_r(int source, int tag, int context, MPI_Status * status) -{ - int rc = FALSE; -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { - if (likely(!MPIDI_Recvq_empty_uexp())) - { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - rc = MPIDI_Recvq_FU(source, tag, context, status); - MPIU_THREAD_CS_EXIT(MSGQUEUE, 0); - } - } - else - { -#endif - if (likely(MPIDI_Recvq.unexpected_head != NULL)) - { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - rc = MPIDI_Recvq_FU(source, tag, context, status); - MPIU_THREAD_CS_EXIT(MSGQUEUE, 0); - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - return rc; -} - - -/** - * \brief Find a request in the unexpected queue and dequeue it, or allocate a new request and enqueue it in the posted queue - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \param[out] foundp TRUE iff the request was found - * \return The matching UE request or the new posted request - */ -#ifndef OUT_OF_ORDER_HANDLING -static inline MPIR_Request * -MPIDI_Recvq_FDU_or_AEP(MPIR_Request *newreq, int source, int tag, int context_id, int * foundp) -#else -static inline MPIR_Request * -MPIDI_Recvq_FDU_or_AEP(MPIR_Request *newreq, int source, pami_task_t pami_source, int tag, int context_id, int * foundp) -#endif -{ - MPIR_Request * rreq = NULL; - /* We have unexpected messages, so search unexpected queue */ -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { - if (unlikely(!MPIDI_Recvq_empty_uexp())) - { -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDU(source, tag, context_id, foundp); -#else - rreq = MPIDI_Recvq_FDU(source, pami_source, tag, context_id, foundp); -#endif - if (*foundp == TRUE) - return rreq; -#if (MPIDI_STATISTICS) - else { - MPID_NSTAT(mpid_statp->lateArrivals); - } -#endif - } - } - else - { -#endif - if (unlikely(MPIDI_Recvq.unexpected_head != NULL)) { -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDU(source, tag, context_id, foundp); -#else - rreq = MPIDI_Recvq_FDU(source, pami_source, tag, context_id, foundp); -#endif - if (*foundp == TRUE) - return rreq; -#if (MPIDI_STATISTICS) - else { - MPID_NSTAT(mpid_statp->lateArrivals); - } -#endif - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - /* A matching request was not found in the unexpected queue, - so we need to allocate a new request and add it to the - posted queue */ - rreq = newreq; - TRACE_SET_REQ_VAL(rreq->mpid.envelope.msginfo.MPIseqno,-1); - TRACE_SET_REQ_VAL(rreq->mpid.envelope.length,-1); - TRACE_SET_REQ_VAL(rreq->mpid.envelope.data,(void *) 0); - rreq->kind = MPIR_REQUEST_KIND__RECV; - MPIDI_Request_setMatch(rreq, tag, source, context_id); -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - MPIDI_Recvq_insert_post(rreq, source, tag, context_id); - else -#endif - MPIDI_Recvq_append(MPIDI_Recvq.posted, rreq); - *foundp = FALSE; - - return rreq; -} - -#if TOKEN_FLOW_CONTROL -typedef struct MPIDI_Token_cntr { - uint16_t unmatched; /* no. of unmatched EA messages */ - uint16_t rettoks; /* no. of tokens to be returned */ - int tokens; /* no. of tokens available-pairwise */ - int n_tokenStarved; /* no. of times token starvation occured */ -} MPIDI_Token_cntr_t; - -MPIDI_Token_cntr_t *MPIDI_Token_cntr; -#endif - -#ifdef OUT_OF_ORDER_HANDLING - -/** - * data structures that tracks pair-wise in-coming communication. - */ -typedef struct MPIDI_In_cntr { - uint n_OutOfOrderMsgs:16; /* the number of out-of-order messages received */ - uint nMsgs; /* the number of received messages */ - MPIR_Request *OutOfOrderList; /* link list of out-of-order messages */ -} MPIDI_In_cntr_t; - -/** - * data structures that tracks pair-wise Out-going communication. - */ -typedef struct MPIDI_Out_cntr { - uint unmatched:16; /* the number of un-matched messages */ - uint nMsgs; /* the number of out-going messages */ -} MPIDI_Out_cntr_t; - -/* global data to keep track of pair-wise communication, storage malloced -during initialization time */ -MPIDI_In_cntr_t *MPIDI_In_cntr; -MPIDI_Out_cntr_t *MPIDI_Out_cntr; - -#endif - - -/** - * \brief Find a request in the posted queue and dequeue it - * \param[in] source Find by Sender - * \param[in] tag Find by Tag - * \param[in] context_id Find by Context ID (communicator) - * \return The matching posted request or the new UE request - */ -#ifndef OUT_OF_ORDER_HANDLING -static inline MPIR_Request * -MPIDI_Recvq_FDP(size_t source, size_t tag, size_t context_id) -#else -static inline MPIR_Request * -MPIDI_Recvq_FDP(size_t source, pami_task_t pami_source, int tag, int context_id, int msg_seqno) -#endif -{ - MPIR_Request * rreq; - MPIR_Request * prev_rreq = NULL; -#ifdef USE_STATISTICS - unsigned search_length = 0; -#endif - TRACE_MEMSET_R(pami_source,msg_seqno,recv_status); - -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - void * it; - if(MPIDI_Process.queue_binary_search_support_on) - { - MPIDI_Recvq_find_in_post(source, tag, context_id, &rreq, &it); - if(rreq == NULL) - { - MPIDI_Recvq_find_in_post(source, MPI_ANY_TAG, context_id, &rreq, &it); - if(rreq == NULL) - { - MPIDI_Recvq_find_in_post(MPI_ANY_SOURCE, tag, context_id, &rreq, &it); - if(rreq == NULL) - MPIDI_Recvq_find_in_post(MPI_ANY_SOURCE, MPI_ANY_TAG, context_id, &rreq, &it); - } - } - } - else -#endif - rreq = MPIDI_Recvq.posted_head; - -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_In_cntr_t *in_cntr = &MPIDI_In_cntr[pami_source]; - int nMsgs=(in_cntr->nMsgs+1); - - if(msg_seqno == nMsgs) { - in_cntr->nMsgs = msg_seqno; - } - - if( ((int)(in_cntr->nMsgs - msg_seqno)) >= 0) { -#endif - while (rreq != NULL) { -#ifdef USE_STATISTICS - ++search_length; -#endif - - int match_src = MPIDI_Request_getMatchRank(rreq); - int match_tag = MPIDI_Request_getMatchTag(rreq); - int match_ctx = MPIDI_Request_getMatchCtxt(rreq); - - int flag0 = (source == match_src); - flag0 |= (match_src == MPI_ANY_SOURCE); - int flag1 = (context_id == match_ctx); - int flag2 = (tag == match_tag); - flag2 |= (match_tag == MPI_ANY_TAG); - int flag = flag0 & flag1 & flag2; - -#if 0 - if ((MPIDI_Request_getMatchRank(rreq)==source || MPIDI_Request_getMatchRank(rreq)==MPI_ANY_SOURCE) && - (MPIDI_Request_getMatchCtxt(rreq)==context_id) && - (MPIDI_Request_getMatchTag(rreq) == tag || MPIDI_Request_getMatchTag(rreq) == MPI_ANY_TAG) - ) -#else - if (flag) -#endif - { - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),req,rreq); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),msgid,msg_seqno); - TRACE_SET_R_BIT(pami_source,(msg_seqno & SEQMASK),fl.f.posted); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),bufadd,rreq->mpid.userbuf); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),len,rreq->mpid.envelope.length); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),rtag,tag); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),rctx,context_id); - TRACE_SET_R_VAL(pami_source,(msg_seqno & SEQMASK),rsource,pami_source); - TRACE_SET_R_BIT(pami_source,(msg_seqno & SEQMASK),fl.f.matchedInHH); - TRACE_SET_REQ_VAL(rreq->mpid.idx,(msg_seqno & SEQMASK)); - TRACE_SET_REQ_VAL(rreq->mpid.partner_id,pami_source); -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_Request_setPeerRank_pami(rreq, pami_source); -#endif -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - MPIDI_Recvq_remove_post(match_src, match_tag, match_ctx, it); - else -#endif - MPIDI_Recvq_remove(MPIDI_Recvq.posted, rreq, prev_rreq); -#ifdef USE_STATISTICS - MPIDI_Statistics_time(MPIDI_Statistics.recvq.unexpected_search, search_length); -#endif - return rreq; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - break; - else - { -#endif - prev_rreq = rreq; - rreq = rreq->mpid.next; -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - } -#ifdef OUT_OF_ORDER_HANDLING - } -#endif - -#ifdef USE_STATISTICS - MPIDI_Statistics_time(MPIDI_Statistics.recvq.unexpected_search, search_length); -#endif - return NULL; -} - - -#ifdef OUT_OF_ORDER_HANDLING -/** - * insert a request _req2 before _req1 - */ -#define MPIDI_Recvq_insert(__Q, __req1, __req2) \ -({ \ - (__req2)->mpid.next = __req1; \ - if((__req1)->mpid.prev != NULL) \ - ((__req1)->mpid.prev)->mpid.next = (__req2); \ - else \ - __Q ## _head = __req2; \ - (__req2)->mpid.prev = (__req1)->mpid.prev; \ - (__req1)->mpid.prev = (__req2); \ -}) - -/** - * remove a request from out of order list - */ -#define MPIDI_Recvq_remove_req_from_ool(req,in_cntr) \ -({ \ - in_cntr->n_OutOfOrderMsgs--; \ - if (in_cntr->n_OutOfOrderMsgs == 0) { \ - in_cntr->OutOfOrderList=NULL; \ - req->mpid.nextR=NULL; \ - req->mpid.prevR=NULL; \ - } else if (in_cntr->n_OutOfOrderMsgs > 0) { \ - in_cntr->OutOfOrderList=req->mpid.nextR; \ - /* remove req from out of order list */ \ - ((MPIR_Request *)(req)->mpid.prevR)->mpid.nextR = (req)->mpid.nextR; \ - ((MPIR_Request *)(req)->mpid.nextR)->mpid.prevR = (req)->mpid.prevR; \ - (req)->mpid.nextR=NULL; \ - (req)->mpid.prevR=NULL; \ - } \ -}) -#endif - - -#endif diff --git a/src/mpid/pamid/src/mpid_recvq_mmap.cpp b/src/mpid/pamid/src/mpid_recvq_mmap.cpp deleted file mode 100644 index c565fed..0000000 --- a/src/mpid/pamid/src/mpid_recvq_mmap.cpp +++ /dev/null @@ -1,419 +0,0 @@ -#include <map> -#include <stdio.h> -#include <cstdlib> -#include "mpidi_platform.h" - - -extern "C" { - -struct MPID_Queue_map_key_t -{ - int context_id; - int source; - int tag; - inline bool operator< (const MPID_Queue_map_key_t& qmk) const - { - if(context_id < qmk.context_id) - return true; - else if(context_id == qmk.context_id) - { - if(source < qmk.source) - return true; - else if(source == qmk.source) - { - if(tag < qmk.tag) - return true; - else - return false; - } - else - return false; - } - else - return false; - } -}; - -struct MPID_Queue_map_value_t -{ - void* rreq; -#ifdef OUT_OF_ORDER_HANDLING - int seqno; -#endif -}; - -typedef std::multimap<MPID_Queue_map_key_t,MPID_Queue_map_value_t> MPID_Req_queue_map_t; -typedef std::multimap<MPID_Queue_map_key_t,MPID_Queue_map_value_t>::iterator MPID_Req_queue_map_iterator_t; - -MPID_Req_queue_map_t MPID_Unexp_queue; -MPID_Req_queue_map_t MPID_Posted_queue; - -MPID_Req_queue_map_iterator_t itp; -MPID_Req_queue_map_iterator_t itu; - -void MPIDI_Recvq_init_queues(); -int MPIDI_Recvq_empty_uexp(); -int MPIDI_Recvq_empty_post(); - -#ifndef OUT_OF_ORDER_HANDLING -void MPIDI_Recvq_insert_uexp(void * rreq, int source, int tag, int context_id); -void MPIDI_Recvq_insert_post(void * rreq, int source, int tag, int context_id); -void MPIDI_Recvq_insrt(MPID_Req_queue_map_t* queue, void * rreq, int source, int tag, int context_id); - -void MPIDI_Recvq_remove_uexp(int source, int tag, int context_id, void* it_req); -void MPIDI_Recvq_remove_post(int source, int tag, int context_id, void* it_req); -void MPIDI_Recvq_rm(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, void* it_req); - -void MPIDI_Recvq_find_in_uexp(int source, int tag, int context_id, void** req, void** it_req); -void MPIDI_Recvq_find_in_post(int source, int tag, int context_id, void** req, void** it_req); -void MPIDI_Recvq_find(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, void** req, void** it_req); - -#else -void MPIDI_Recvq_insert_uexp(void * rreq, int source, int tag, int context_id, int seqno); -void MPIDI_Recvq_insert_post(void * rreq, int source, int tag, int context_id); -void MPIDI_Recvq_insrt(MPID_Req_queue_map_t* queue, void * rreq, int source, int tag, int context_id, int seqno); - -void MPIDI_Recvq_remove_uexp(int source, int tag, int context_id, int seqno, void* it_req); -void MPIDI_Recvq_remove_uexp_noit(int source, int tag, int context_id, int seqno); -void MPIDI_Recvq_remove_post(int source, int tag, int context_id, void* it_req); -void MPIDI_Recvq_rm(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, int seqno, void* it_req); - -void MPIDI_Recvq_find_in_uexp(int source, int tag, int context_id, int seqno, void** req, void** it_req); -void MPIDI_Recvq_find_in_post(int source, int tag, int context_id, void** req, void** it_req); -void MPIDI_Recvq_find(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, int seqno, void** req, void** it_req); - -#endif - - - - -void MPIDI_Recvq_init_queues() -{ - MPID_Unexp_queue.clear(); - MPID_Posted_queue.clear(); -} - -int MPIDI_Recvq_empty_uexp() -{ - return MPID_Unexp_queue.empty(); -} - -int MPIDI_Recvq_empty_post() -{ - return MPID_Posted_queue.empty(); -} - -#ifndef OUT_OF_ORDER_HANDLING -void MPIDI_Recvq_insert_uexp(void * rreq, int source, int tag, int context_id) -{ - MPIDI_Recvq_insrt(&MPID_Unexp_queue, rreq, source, tag, context_id); -} - - -void MPIDI_Recvq_insert_post(void * rreq, int source, int tag, int context_id) -{ - MPIDI_Recvq_insrt(&MPID_Posted_queue, rreq, source, tag, context_id); -} - - -void MPIDI_Recvq_insrt(MPID_Req_queue_map_t* queue, void * rreq, int source, int tag, int context_id) -{ - MPID_Queue_map_key_t key; - MPID_Queue_map_value_t value; - key.context_id = context_id; - key.source = source; - key.tag = tag; - value.rreq = rreq; - queue->insert(std::make_pair(key,value)); -} - - -void MPIDI_Recvq_remove_uexp(int source, int tag, int context_id, void* it_req) -{ - MPIDI_Recvq_rm(&MPID_Unexp_queue, source, tag, context_id, it_req); -} - - -void MPIDI_Recvq_remove_post(int source, int tag, int context_id, void* it_req) -{ - MPIDI_Recvq_rm(&MPID_Posted_queue, source, tag, context_id, it_req); -} - - -void MPIDI_Recvq_rm(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, void* it_req) -{ - MPID_Req_queue_map_iterator_t it = *((MPID_Req_queue_map_iterator_t*)it_req); - queue->erase(it); -} - - -void MPIDI_Recvq_find_in_uexp(int source, int tag, int context_id, void** req, void** it_req) -{ - return MPIDI_Recvq_find(&MPID_Unexp_queue, source, tag, context_id, req, it_req); -} - -void MPIDI_Recvq_find_in_post(int source, int tag, int context_id, void** req, void** it_req) -{ - MPID_Queue_map_key_t key; - key.context_id = context_id; - key.source = source; - key.tag = tag; - *it_req = NULL; - *req = NULL; - itp = MPID_Posted_queue.find(key); - if(itp != MPID_Posted_queue.end()) - { - *it_req = &itp; - *req = ((MPID_Queue_map_value_t)(itp->second)).rreq; - } -} - -void MPIDI_Recvq_find(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, void** req, void** it_req) -{ - MPID_Queue_map_key_t key; - key.context_id = context_id; - key.source = source; - key.tag = tag; - *req = NULL; - *it_req = NULL; - itu = queue->find(key); - if(itu != queue->end()) - { - *req = ((MPID_Queue_map_value_t)(itu->second)).rreq; - *it_req = &itu; - return; - } - - if(source < 0 && tag >= 0) - { - for(itu = queue->begin(); itu != queue->end(); itu++) - { - if(((MPID_Queue_map_key_t)itu->first).tag == tag && ((MPID_Queue_map_key_t)itu->first).context_id == context_id) - { - *it_req = &itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - } - } - else if(source >= 0 && tag < 0) - { - for(itu = queue->begin(); itu != queue->end(); itu++) - { - if(((MPID_Queue_map_key_t)itu->first).source == source && ((MPID_Queue_map_key_t)itu->first).context_id == context_id) - { - *it_req = &itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - } - } - else if(source < 0 && tag < 0) - { - for(itu = queue->begin(); itu != queue->end(); itu++) - { - if(((MPID_Queue_map_key_t)itu->first).context_id == context_id) - { - *it_req = &itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - } - } - -} - -#else -void MPIDI_Recvq_insert_uexp(void * rreq, int source, int tag, int context_id, int seqno) -{ - MPIDI_Recvq_insrt(&MPID_Unexp_queue, rreq, source, tag, context_id, seqno); - -} - - -void MPIDI_Recvq_insert_post(void * rreq, int source, int tag, int context_id) -{ - MPIDI_Recvq_insrt(&MPID_Posted_queue, rreq, source, tag, context_id, -1); -} - - -void MPIDI_Recvq_insrt(MPID_Req_queue_map_t* queue, void * rreq, int source, int tag, int context_id, int seqno) -{ - MPID_Queue_map_key_t key; - MPID_Queue_map_value_t value; - key.context_id = context_id; - key.source = source; - key.tag = tag; - value.seqno = seqno; - value.rreq = rreq; - queue->insert(std::make_pair(key,value)); -} - - -void MPIDI_Recvq_remove_uexp(int source, int tag, int context_id, int seqno, void* it_req) -{ - MPIDI_Recvq_rm(&MPID_Unexp_queue, source, tag, context_id, seqno, it_req); -} - -void MPIDI_Recvq_remove_uexp_noit(int source, int tag, int context_id, int seqno) -{ - MPID_Queue_map_key_t key; - key.context_id = context_id; - key.source = source; - key.tag = tag; - MPID_Req_queue_map_iterator_t it; - std::pair <MPID_Req_queue_map_iterator_t, MPID_Req_queue_map_iterator_t > itpair; - itpair = MPID_Unexp_queue.equal_range(key); - for(it = itpair.first; itu != itpair.second; ++it) - if(((MPID_Queue_map_value_t)it->second).seqno == seqno) - { - MPID_Unexp_queue.erase(it); - break; - } -} - -void MPIDI_Recvq_remove_post(int source, int tag, int context_id, void* it_req) -{ - MPIDI_Recvq_rm(&MPID_Posted_queue, source, tag, context_id, -1, it_req); -} - - -void MPIDI_Recvq_rm(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, int seqno, void* it_req) -{ - MPID_Req_queue_map_iterator_t it = *((MPID_Req_queue_map_iterator_t*)it_req); - queue->erase(it); -} - - -void MPIDI_Recvq_find_in_uexp(int source, int tag, int context_id, int seqno, void** req, void** it_req) -{ - return MPIDI_Recvq_find(&MPID_Unexp_queue, source, tag, context_id, seqno, req, it_req); -} - -void MPIDI_Recvq_find_in_post(int source, int tag, int context_id, void** req, void** it_req) -{ - MPID_Queue_map_key_t key; - key.context_id = context_id; - key.source = source; - key.tag = tag; - *req = NULL; - *it_req = NULL; - itp = MPID_Posted_queue.find(key); - if(itp!=MPID_Posted_queue.end()) - { - *it_req = (void*)&itp; - *req = ((MPID_Queue_map_value_t)(itp->second)).rreq; - } -} - - -void MPIDI_Recvq_find(MPID_Req_queue_map_t* queue, int source, int tag, int context_id, int seqno, void** req, void** it_req) -{ - MPID_Queue_map_key_t key; - key.context_id = context_id; - key.source = source; - key.tag = tag; - - *req = NULL; - *it_req = NULL; - if(seqno == -1) - { - itu = queue->find(key); - if(itu != queue->end()) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - } - } - else - { - std::pair <MPID_Req_queue_map_iterator_t, MPID_Req_queue_map_iterator_t > itpair; - itpair = queue->equal_range(key); - for(itu = itpair.first; itu != itpair.second; ++itu) - if(((MPID_Queue_map_value_t)itu->second).seqno <= seqno) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - break; - } - } - if(*req != NULL) - return; - - if(source < 0 && tag >= 0) - { - for(itu = queue->begin(); itu != queue->end(); itu++) - { - if(((MPID_Queue_map_key_t)itu->first).tag == tag && ((MPID_Queue_map_key_t)itu->first).context_id == context_id) - { - if(seqno == -1) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - else - { - if(((MPID_Queue_map_value_t)itu->second).seqno <= seqno) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - } - } - } - } - else if(source >= 0 && tag < 0) - { - for(itu = queue->begin(); itu != queue->end(); itu++) - { - if(((MPID_Queue_map_key_t)itu->first).source == source && ((MPID_Queue_map_key_t)itu->first).context_id == context_id) - { - if(seqno == -1) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - else - { - if(((MPID_Queue_map_value_t)itu->second).seqno <= seqno) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - } - } - } - } - else if(source < 0 && tag < 0) - { - for(itu = queue->begin(); itu != queue->end(); itu++) - { - if(((MPID_Queue_map_key_t)itu->first).context_id == context_id) - { - if(seqno == -1) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - else - { - if(((MPID_Queue_map_value_t)itu->second).seqno <= seqno) - { - *it_req = (void*)&itu; - *req = ((MPID_Queue_map_value_t)itu->second).rreq; - return; - } - } - } - } - } -} - -#endif - -} diff --git a/src/mpid/pamid/src/mpid_request.c b/src/mpid/pamid/src/mpid_request.c deleted file mode 100644 index 0a1607d..0000000 --- a/src/mpid/pamid/src/mpid_request.c +++ /dev/null @@ -1,88 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_request.c - * \brief Accessors and actors for MPID Requests - */ -#include <mpidimpl.h> - -#ifndef MPIR_REQUEST_PREALLOC -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__GLOBAL) -#define MPIR_REQUEST_PREALLOC 16 -#elif (MPIU_HANDLE_ALLOCATION_METHOD == MPIU_HANDLE_ALLOCATION_THREAD_LOCAL) -#define MPIR_REQUEST_PREALLOC 512 //Have direct more reqyests for all threads -#else -#define MPIR_REQUEST_PREALLOC 8 -#endif -#endif - -/** - * \defgroup MPIR_REQUEST MPID Request object management - * - * Accessors and actors for MPID Requests - */ - - -/* these are referenced by src/mpi/pt2pt/wait.c in PMPI_Wait! */ -MPIR_Request MPIR_Request_direct[MPIR_REQUEST_PREALLOC] __attribute__((__aligned__(64))); -MPIR_Object_alloc_t MPIR_Request_mem = - { - 0, 0, 0, 0, MPIR_REQUEST, sizeof(MPIR_Request), - MPIR_Request_direct, - MPIR_REQUEST_PREALLOC - }; - - -#if (MPIU_HANDLE_ALLOCATION_METHOD == MPIU_HANDLE_ALLOCATION_THREAD_LOCAL) && defined(__BGQ__) -void MPIDI_Request_allocate_pool() -{ - int i; - MPIR_Request *prev, *cur; - /* batch allocate a linked list of requests */ - MPIU_THREAD_CS_ENTER(HANDLEALLOC,); - prev = MPIR_Handle_obj_alloc_unsafe(&MPIR_Request_mem); - MPID_assert(prev != NULL); - prev->mpid.next = NULL; - for (i = 1; i < MPIR_REQUEST_TLS_MAX; ++i) { - cur = MPIR_Handle_obj_alloc_unsafe(&MPIR_Request_mem); - MPID_assert(cur != NULL); - cur->mpid.next = prev; - prev = cur; - } - MPIU_THREAD_CS_EXIT(HANDLEALLOC,); - MPIDI_Process.request_handles[MPIDI_THREAD_ID()].head = cur; - MPIDI_Process.request_handles[MPIDI_THREAD_ID()].count += MPIR_REQUEST_TLS_MAX; -} -#endif - - -void -MPIDI_Request_uncomplete(MPIR_Request *req) -{ - int count; - MPIR_Object_add_ref(req); - MPIR_cc_incr(req->cc_ptr, &count); -} - - -void -MPID_Request_set_completed(MPIR_Request *req) -{ - MPIR_cc_set(&req->cc, 0); - MPIDI_Progress_signal(); -} diff --git a/src/mpid/pamid/src/mpid_request.h b/src/mpid/pamid/src/mpid_request.h deleted file mode 100644 index ad283cf..0000000 --- a/src/mpid/pamid/src/mpid_request.h +++ /dev/null @@ -1,354 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_request.h - * \brief ??? - */ - -#ifndef __src_mpid_request_h__ -#define __src_mpid_request_h__ - -#include "mpidu_datatype.h" - -/** - * \addtogroup MPIR_REQUEST - * \{ - */ - -#define MPIR_Request_create MPID_Request_create_inline -#define MPIR_Request_free MPID_Request_free_inline -#define MPIDI_Request_complete MPIDI_Request_complete_inline -#define MPIDI_Request_complete_norelease MPIDI_Request_complete_norelease_inline -#define MPID_Request_discard MPID_Request_discard_inline - - -extern MPIR_Object_alloc_t MPIR_Request_mem; -#if TOKEN_FLOW_CONTROL -extern void MPIDI_mm_free(void *,size_t); -#endif -typedef enum {mpiuMalloc=1,mpidiBufMM} MPIDI_mallocType; - -void MPIDI_Request_uncomplete(MPIR_Request *req); -#if (MPIU_HANDLE_ALLOCATION_METHOD == MPIU_HANDLE_ALLOCATION_THREAD_LOCAL) && defined(__BGQ__) -void MPIDI_Request_allocate_pool(); -#endif - -#define MPIDI_Request_getCA(_req) ({ (_req)->mpid.ca; }) -#define MPIDI_Request_getPeerRank_pami(_req) ({ (_req)->mpid.peer_pami; }) -#define MPIDI_Request_getPeerRank_comm(_req) ({ (_req)->mpid.peer_comm; }) -#define MPIDI_Request_getPType(_req) ({ (_req)->mpid.ptype; }) -#define MPIDI_Request_getControl(_req) ({ (_req)->mpid.envelope.msginfo.control; }) -#define MPIDI_Request_isSync(_req) ({ (_req)->mpid.envelope.msginfo.isSync; }) -#define MPIDI_Request_isRzv(_req) ({ (_req)->mpid.envelope.msginfo.isRzv; }) -#define MPIDI_Request_getMatchTag(_req) ({ (_req)->mpid.envelope.msginfo.MPItag; }) -#define MPIDI_Request_getMatchRank(_req) ({ (_req)->mpid.envelope.msginfo.MPIrank; }) -#define MPIDI_Request_getMatchCtxt(_req) ({ (_req)->mpid.envelope.msginfo.MPIctxt; }) - -#define MPIDI_Request_setCA(_req, _ca) ({ (_req)->mpid.ca = (_ca); }) -#define MPIDI_Request_setPeerRank_pami(_req,_r) ({ (_req)->mpid.peer_pami = (_r); }) -#define MPIDI_Request_setPeerRank_comm(_req,_r) ({ (_req)->mpid.peer_comm = (_r); }) -#define MPIDI_Request_setPType(_req,_t) ({ (_req)->mpid.ptype = (_t); }) -#define MPIDI_Request_setControl(_req,_t) ({ (_req)->mpid.envelope.msginfo.control = (_t); }) -#define MPIDI_Request_setSync(_req,_t) ({ (_req)->mpid.envelope.msginfo.isSync = (_t); }) -#define MPIDI_Request_setRzv(_req,_t) ({ (_req)->mpid.envelope.msginfo.isRzv = (_t); }) -#ifdef OUT_OF_ORDER_HANDLING -#define MPIDI_Request_getMatchSeq(_req) ({ (_req)->mpid.envelope.msginfo.MPIseqno; }) -#define MPIDI_Request_setMatchSeq(_req,_sq) ({ (_req)->mpid.envelope.msginfo.MPIseqno = (_sq); }) -#endif -#define MPIDI_Request_setMatch(_req,_tag,_rank,_ctxtid) \ -({ \ - (_req)->mpid.envelope.msginfo.MPItag=(_tag); \ - (_req)->mpid.envelope.msginfo.MPIrank=(_rank); \ - (_req)->mpid.envelope.msginfo.MPIctxt=(_ctxtid); \ -}) - -#define MPIDI_Msginfo_getPeerRequest(_msg) ({ MPIR_Request *req=NULL; MPIR_Request_get_ptr((_msg)->req, req); MPID_assert(req != NULL); req; }) -#define MPIDI_Msginfo_getPeerRequestH(_msg) ({ (_msg)->req; }) -#define MPIDI_Msginfo_cpyPeerRequestH(_dst,_src) ({ (_dst)->req = (_src)->req; MPI_SUCCESS; }) -#define MPIDI_Request_getPeerRequest(_req) MPIDI_Msginfo_getPeerRequest(&(_req)->mpid.envelope.msginfo) -#define MPIDI_Request_getPeerRequestH(_req) ({ (_req)->mpid.envelope.msginfo.req; }) -#define MPIDI_Request_setPeerRequestH(_req) ({ (_req)->mpid.envelope.msginfo.req = (_req)->handle; MPI_SUCCESS; }) -#define MPIDI_Request_cpyPeerRequestH(_dst,_src) MPIDI_Msginfo_cpyPeerRequestH(&(_dst)->mpid.envelope.msginfo,_src) - - -#define MPIU_HANDLE_ALLOCATION_MUTEX 0 -#define MPIU_HANDLE_ALLOCATION_THREAD_LOCAL 1 - -/* XXX DJG for TLS hack */ -#define MPIR_REQUEST_TLS_MAX 128 - -#if (MPIU_HANDLE_ALLOCATION_METHOD == MPIU_HANDLE_ALLOCATION_THREAD_LOCAL) && defined(__BGQ__) - -# define MPIDI_Request_tls_alloc(req) \ -({ \ - size_t tid = MPIDI_THREAD_ID(); \ - MPIDI_RequestHandle_t *rh = &MPIDI_Process.request_handles[tid]; \ - if (unlikely(rh->head == NULL)) \ - MPIDI_Request_allocate_pool(); \ - (req) = rh->head; \ - rh->head = req->mpid.next; \ - rh->count --; \ -}) - -# define MPIDI_Request_tls_free(req) \ -({ \ - size_t tid = MPIDI_THREAD_ID(); \ - MPIDI_RequestHandle_t *rh = &MPIDI_Process.request_handles[tid]; \ - if (likely(rh->count < MPIR_REQUEST_TLS_MAX)) \ - { \ - /* push request onto the top of the stack */ \ - req->mpid.next = rh->head; \ - rh->head = req; \ - rh->count ++; \ - } \ - else \ - { \ - MPIR_Handle_obj_free(&MPIR_Request_mem, req); \ - } \ -}) - -#else - -# define MPIDI_Request_tls_alloc(req) \ -({ \ - (req) = MPIR_Handle_obj_alloc(&MPIR_Request_mem); \ - if (req == NULL) \ - MPID_Abort(NULL, MPI_ERR_NO_SPACE, -1, "Cannot allocate Request"); \ -}) - -# define MPIDI_Request_tls_free(req) MPIR_Handle_obj_free(&MPIR_Request_mem, (req)) - -#endif - -#ifdef HAVE_DEBUGGER_SUPPORT -#define MPIDI_Request_clear_dbg(req_) ((req_)->u.send.dbg_next = NULL) -#else -#define MPIDI_Request_clear_dbg(req_) -#endif - -/** - * \brief Create a very generic request - * \note This should only ever be called by more specific allocators - */ -static inline MPIR_Request * -MPIDI_Request_create_basic() -{ - MPIR_Request * req = NULL; - - MPIDI_Request_tls_alloc(req); - MPID_assert(req != NULL); - MPID_assert(HANDLE_GET_MPI_KIND(req->handle) == MPIR_REQUEST); - MPIR_cc_set(&req->cc, 1); - req->cc_ptr = &req->cc; - -#if 0 - /* This will destroy the MPID part of the request. Use this to - check for fields that are not being correctly initialized. */ - memset(&req->mpid, 0xFFFFFFFF, sizeof(struct MPIDI_Request)); -#endif - req->mpid.next = NULL; - MPIDI_Request_clear_dbg(req); - - return req; -} - - -/** - * \brief Create new request without initalizing - */ -static inline MPIR_Request * -MPIDI_Request_create2_fast() -{ - MPIR_Request * req; - req = MPIDI_Request_create_basic(); - MPIR_Object_set_ref(req, 2); - - return req; -} - - -/** - * \brief Create and initialize a new request - */ -static inline void -MPIDI_Request_initialize(MPIR_Request * req) -{ - req->greq_fns = NULL; - - MPIR_STATUS_SET_COUNT(req->status, 0); - MPIR_STATUS_SET_CANCEL_BIT(req->status, FALSE); - req->status.MPI_SOURCE = MPI_UNDEFINED; - req->status.MPI_TAG = MPI_UNDEFINED; - req->status.MPI_ERROR = MPI_SUCCESS; - - struct MPIDI_Request* mpid = &req->mpid; - mpid->envelope.msginfo.flags = 0; - mpid->cancel_pending = FALSE; - mpid->datatype_ptr = NULL; - mpid->uebuf = NULL; - mpid->uebuflen = 0; - mpid->uebuf_malloc = 0; -#ifdef OUT_OF_ORDER_HANDLING - mpid->prev = NULL; - mpid->prevR = NULL; - mpid->nextR = NULL; - mpid->oo_peer = NULL; -#endif - mpid->win_req = NULL; - MPIDI_Request_setCA(req, MPIDI_CA_COMPLETE); -} - - -/** - * \brief Create and initialize a new request - */ -static inline MPIR_Request * -MPID_Request_create_inline() -{ - MPIR_Request * req; - req = MPIDI_Request_create_basic(); - MPIR_Object_set_ref(req, 1); - - MPIDI_Request_initialize(req); - req->comm=NULL; - - return req; -} - - -/** - * \brief Create and initialize a new request - */ -static inline MPIR_Request * -MPIDI_Request_create2() -{ - MPIR_Request * req; - req = MPIR_Request_create(); - MPIR_Object_set_ref(req, 2); - - return req; -} - -static inline MPIR_Request * -MPIDI_Request_create1() -{ - MPIR_Request * req; - req = MPIR_Request_create(); - MPIR_Object_set_ref(req, 1); - - return req; -} - -/** - * \brief Mark a request as cancel-pending - * \param[in] _req The request to cancel - * \return The previous state - */ -#define MPIDI_Request_cancel_pending(_req) \ -({ \ - int _flag = (_req)->mpid.cancel_pending; \ - (_req)->mpid.cancel_pending = TRUE; \ - _flag; \ -}) - - -static inline void -MPID_Request_free_inline(MPIR_Request *req) -{ - int count; - MPID_assert(HANDLE_GET_MPI_KIND(req->handle) == MPIR_REQUEST); - MPIR_Object_release_ref(req, &count); - MPID_assert(count >= 0); - - - if (count == 0) - { - MPID_assert(MPIR_cc_is_complete(&req->cc)); - - if (req->comm) MPIR_Comm_release(req->comm, 0); - if (req->greq_fns) MPL_free(req->greq_fns); - if (req->mpid.datatype_ptr) MPIDU_Datatype_release(req->mpid.datatype_ptr); - if (req->mpid.uebuf_malloc== mpiuMalloc) { - MPL_free(req->mpid.uebuf); - } - if(req->mpid.win_req) MPL_free(req->mpid.win_req); -#if TOKEN_FLOW_CONTROL - else if (req->mpid.uebuf_malloc == mpidiBufMM) { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - MPIDI_mm_free(req->mpid.uebuf,req->mpid.uebuflen); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } -#endif - MPIDI_Request_tls_free(req); - } -} - - -/* This request was never used, at most had uebuf allocated. */ -static inline void -MPID_Request_discard_inline(MPIR_Request *req) -{ - if (req->mpid.uebuf_malloc == mpiuMalloc) { - MPL_free(req->mpid.uebuf); - } -#if TOKEN_FLOW_CONTROL - else if (req->mpid.uebuf_malloc == mpidiBufMM) { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - MPIDI_mm_free(req->mpid.uebuf,req->mpid.uebuflen); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } -#endif - MPIDI_Request_tls_free(req); -} - -#define MPIR_REQUEST_SET_COMPLETED(req_) \ - MPIDI_Request_complete_norelease_inline(req_) - -static inline void -MPIDI_Request_complete_inline(MPIR_Request *req) -{ - int count; - MPIR_cc_decr(req->cc_ptr, &count); - MPID_assert(count >= 0); - - MPIR_Request_free(req); - if (count == 0) /* decrement completion count; if 0, signal progress engine */ - { - MPIDI_Progress_signal(); - } -} - - -static inline void -MPIDI_Request_complete_norelease_inline(MPIR_Request *req) -{ - int count; - MPIR_cc_decr(req->cc_ptr, &count); - MPID_assert(count >= 0); - - if (count == 0) /* decrement completion count; if 0, signal progress engine */ - { - MPIDI_Progress_signal(); - } -} - - -/** \} */ - - -#endif diff --git a/src/mpid/pamid/src/mpid_time.c b/src/mpid/pamid/src/mpid_time.c deleted file mode 100644 index 95f3856..0000000 --- a/src/mpid/pamid/src/mpid_time.c +++ /dev/null @@ -1,112 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_time.c - * \brief Devince interface between MPI_Wtime() and PAMI_Wtime() - */ -#include <mpidimpl.h> - -int MPIDI_PAMID_Timer_is_ready = 0; - -static int wtime(MPL_time_t *tval) -{ - if (MPIDI_PAMID_Timer_is_ready) { - *((MPID_Time_t *) tval) = PAMI_Wtime(MPIDI_Client); - return MPID_TIMER_SUCCESS; - } - else - return MPID_TIMER_ERR_NOT_INITIALIZED; -} - -int MPID_Wtime(MPID_Time_t *tval) -{ - return wtime((MPL_time_t *) tval); -} - -static int wtick(double *wtick) -{ - if (MPIDI_PAMID_Timer_is_ready) { - *((double *) wtick) = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_WTICK).value.doubleval; - return MPID_TIMER_SUCCESS; - } - else - return MPID_TIMER_ERR_NOT_INITIALIZED; -} - -int MPID_Wtick(double *tick) -{ - return wtick((MPL_time_t *) tick); -} - -static int wtime_diff(MPL_time_t *t1, MPL_time_t *t2, double *diff) -{ - if (MPIDI_PAMID_Timer_is_ready) { - *diff = *((MPID_Time_t *) t2) - *((MPID_Time_t *) t1); - return MPID_TIMER_SUCCESS; - } - else - return MPID_TIMER_ERR_NOT_INITIALIZED; -} - -int MPID_Wtime_diff(MPID_Time_t *t1, MPID_Time_t *t2, double *diff) -{ - return wtime_diff((MPL_time_t *) t1, (MPL_time_t *) t2, diff); -} - -static int wtime_todouble(MPL_time_t *t, double *val) -{ - if (MPIDI_PAMID_Timer_is_ready) { - *val = *((MPID_Time_t *) t); - return MPID_TIMER_SUCCESS; - } - else - return MPID_TIMER_ERR_NOT_INITIALIZED; -} - -int MPID_Wtime_todouble(MPID_Time_t *t, double *val) -{ - return wtime_todouble((MPL_time_t *) t, val); -} - -static int wtime_acc(MPL_time_t *t1, MPL_time_t *t2, MPL_time_t *t3) -{ - if (MPIDI_PAMID_Timer_is_ready) { - *((MPID_Time_t *) t3) += *((MPID_Time_t *) t1) - *((MPID_Time_t *) t2); - return MPID_TIMER_SUCCESS; - } - else - return MPID_TIMER_ERR_NOT_INITIALIZED; -} - -int MPID_Wtime_acc(MPID_Time_t *t1, MPID_Time_t *t2, MPID_Time_t *t3) -{ - return wtime_acc((MPL_time_t *) t1, (MPL_time_t *) t2, (MPL_time_t *) t3); -} - -int MPID_Wtime_init( void ) -{ - MPL_wtime_fn = wtime; - MPL_wtick_fn = wtick; - MPL_wtime_diff_fn = wtime_diff; - MPL_wtime_todouble_fn = wtime_todouble; - MPL_wtime_acc_fn = wtime_acc; - - return MPID_TIMER_SUCCESS; -} - -#endif diff --git a/src/mpid/pamid/src/mpid_vc.c b/src/mpid/pamid/src/mpid_vc.c deleted file mode 100644 index fed17b7..0000000 --- a/src/mpid/pamid/src/mpid_vc.c +++ /dev/null @@ -1,360 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_vc.c - * \brief Maintain the virtual connection reference table - */ -/* - * (C) 2001 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ - -#include <mpidimpl.h> - -extern int mpidi_dynamic_tasking; -/** - * \brief Virtual connection reference table - */ -struct MPIDI_VCRT -{ - MPIR_OBJECT_HEADER; - unsigned size; /**< Number of entries in the table */ - MPID_VCR *vcr_table; /**< Array of virtual connection references */ -}; - - -int MPID_VCR_Dup(MPID_VCR orig_vcr, MPID_VCR * new_vcr) -{ -#ifdef DYNAMIC_TASKING - if(mpidi_dynamic_tasking) { - if (orig_vcr->pg) { - MPIDI_PG_add_ref( orig_vcr->pg ); - } - } -#endif - - (*new_vcr)->taskid = orig_vcr->taskid; -#ifdef DYNAMIC_TASKING - (*new_vcr)->pg_rank = orig_vcr->pg_rank; - (*new_vcr)->pg = orig_vcr->pg; -#endif - return MPI_SUCCESS; -} - -int MPID_VCR_Get_lpid(MPID_VCR vcr, int * lpid_ptr) -{ - *lpid_ptr = (int)(vcr->taskid); - return MPI_SUCCESS; -} - -int MPID_VCRT_Create(int size, MPID_VCRT *vcrt_ptr) -{ - struct MPIDI_VCRT * vcrt; - int i,result; - - vcrt = MPL_malloc(sizeof(struct MPIDI_VCRT)); - vcrt->vcr_table = MPL_malloc(size*sizeof(MPID_VCR)); - - for(i = 0; i < size; i++) { - vcrt->vcr_table[i] = MPL_malloc(sizeof(struct MPID_VCR_t)); - } - if (vcrt != NULL) - { - MPIR_Object_set_ref(vcrt, 1); - vcrt->size = size; - *vcrt_ptr = vcrt; - result = MPI_SUCCESS; - } - else - { - result = MPIR_ERR_MEMALLOCFAILED; - } - return result; -} - -int MPID_VCRT_Add_ref(MPID_VCRT vcrt) -{ - MPIR_Object_add_ref(vcrt); - return MPI_SUCCESS; -} - -int MPID_VCRT_Release(MPID_VCRT vcrt, int isDisconnect) -{ - int count, i; - - MPIR_Object_release_ref(vcrt, &count); - - if (count == 0) { -#ifdef DYNAMIC_TASKING - int inuse; - if(mpidi_dynamic_tasking) { - for (i = 0; i < vcrt->size; i++) - { - MPID_VCR const vcr = vcrt->vcr_table[i]; - - if (vcr->pg == MPIDI_Process.my_pg && - vcr->pg_rank == MPIDI_Process.my_pg_rank) - { - TRACE_ERR("before MPIDI_PG_release_ref on vcr=%x pg=%x pg=%s inuse=%d\n", vcr, vcr->pg, (vcr->pg)->id, inuse); - inuse=MPIR_Object_get_ref(vcr->pg); - MPIDI_PG_release_ref(vcr->pg, &inuse); - if (inuse == 0) - { - MPIDI_PG_Destroy(vcr->pg); - MPL_free(vcr); - } - continue; - } - inuse=MPIR_Object_get_ref(vcr->pg); - - MPIDI_PG_release_ref(vcr->pg, &inuse); - if (inuse == 0) - MPIDI_PG_Destroy(vcr->pg); - if(vcr) MPL_free(vcr); - } - MPL_free(vcrt->vcr_table); - } /** CHECK */ - else { - for (i = 0; i < vcrt->size; i++) - MPL_free(vcrt->vcr_table[i]); - MPL_free(vcrt->vcr_table);vcrt->vcr_table=NULL; - } -#else - for (i = 0; i < vcrt->size; i++) - MPL_free(vcrt->vcr_table[i]); - MPL_free(vcrt->vcr_table);vcrt->vcr_table=NULL; -#endif - MPL_free(vcrt);vcrt=NULL; - } - return MPI_SUCCESS; -} - -int MPID_VCRT_Get_ptr(MPID_VCRT vcrt, MPID_VCR **vc_pptr) -{ - *vc_pptr = vcrt->vcr_table; - return MPI_SUCCESS; -} - -#ifdef DYNAMIC_TASKING -int MPID_VCR_CommFromLpids( MPIR_Comm *newcomm_ptr, - int size, const int lpids[] ) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Comm *commworld_ptr; - int i; - MPIDI_PG_iterator iter; - - commworld_ptr = MPIR_Process.comm_world; - /* Setup the communicator's vc table: remote group */ - MPID_VCRT_Create( size, &newcomm_ptr->vcrt ); - MPID_VCRT_Get_ptr( newcomm_ptr->vcrt, &newcomm_ptr->vcr ); - if(mpidi_dynamic_tasking) { - for (i=0; i<size; i++) { - MPID_VCR vc = 0; - - /* For rank i in the new communicator, find the corresponding - virtual connection. For lpids less than the size of comm_world, - we can just take the corresponding entry from comm_world. - Otherwise, we need to search through the process groups. - */ - /* printf( "[%d] Remote rank %d has lpid %d\n", - MPIR_Process.comm_world->rank, i, lpids[i] ); */ -#if 0 - if (lpids[i] < commworld_ptr->remote_size) { - vc = commworld_ptr->vcr[lpids[i]]; - } - else { -#endif - /* We must find the corresponding vcr for a given lpid */ - /* For now, this means iterating through the process groups */ - MPIDI_PG_t *pg = 0; - int j; - - MPIDI_PG_Get_iterator(&iter); - /* Skip comm_world */ - /*MPIDI_PG_Get_next( &iter, &pg ); */ - do { - MPIDI_PG_Get_next( &iter, &pg ); - /*MPIU_ERR_CHKINTERNAL(!pg, mpi_errno, "no pg"); */ - /* FIXME: a quick check on the min/max values of the lpid - for this process group could help speed this search */ - for (j=0; j<pg->size; j++) { - /*printf( "Checking lpid %d against %d in pg %s\n", - lpids[i], pg->vct[j].taskid, (char *)pg->id ); - fflush(stdout); */ - if (pg->vct[j].taskid == lpids[i]) { - vc = &pg->vct[j]; - /*printf( "found vc %x for lpid = %d in another pg\n", - (int)vc, lpids[i] );*/ - break; - } - } - } while (!vc); -#if 0 - } -#endif - - /* printf( "about to dup vc %x for lpid = %d in another pg\n", - (int)vc, lpids[i] ); */ - /* Note that his will increment the ref count for the associate - PG if necessary. */ - MPID_VCR_Dup( vc, &newcomm_ptr->vcr[i] ); - } - } else { - for (i=0; i<size; i++) { - /* For rank i in the new communicator, find the corresponding - rank in the comm world (FIXME FOR MPI2) */ - /* printf( "[%d] Remote rank %d has lpid %d\n", - MPIR_Process.comm_world->rank, i, lpids[i] ); */ - if (lpids[i] < commworld_ptr->remote_size) { - MPID_VCR_Dup( commworld_ptr->vcr[lpids[i]], - &newcomm_ptr->vcr[i] ); - } - else { - /* We must find the corresponding vcr for a given lpid */ - /* FIXME: Error */ - return 1; - /* MPID_VCR_Dup( ???, &newcomm_ptr->vcr[i] ); */ - } - } - - } -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -/* - * The following is a very simple code for looping through - * the GPIDs. Note that this code requires that all processes - * have information on the process groups. - */ -int MPID_GPID_ToLpidArray( int size, int gpid[], int lpid[] ) -{ - int i, mpi_errno = MPI_SUCCESS; - int pgid; - MPIDI_PG_t *pg = 0; - MPIDI_PG_iterator iter; - - if(mpidi_dynamic_tasking) { - for (i=0; i<size; i++) { - MPIDI_PG_Get_iterator(&iter); - do { - MPIDI_PG_Get_next( &iter, &pg ); - if (!pg) { - /* Internal error. This gpid is unknown on this process */ - TRACE_ERR("No matching pg foung for id = %d\n", pgid ); - lpid[i] = -1; - /*MPIU_ERR_SET2(mpi_errno,MPI_ERR_INTERN, "**unknowngpid", - "**unknowngpid %d %d", gpid[0], gpid[1] ); */ - return mpi_errno; - } - MPIDI_PG_IdToNum( pg, &pgid ); - - if (pgid == gpid[0]) { - /* found the process group. gpid[1] is the rank in - this process group */ - /* Sanity check on size */ - TRACE_ERR("found the progress group for id = %d\n", pgid ); - TRACE_ERR("pg->size = %d gpid[1]=%d\n", pg->size, gpid[1] ); - if (pg->size > gpid[1]) { - TRACE_ERR("pg->vct[gpid[1]].taskid = %d\n", pg->vct[gpid[1]].taskid ); - lpid[i] = pg->vct[gpid[1]].taskid; - } - else { - lpid[i] = -1; - /*MPIU_ERR_SET2(mpi_errno,MPI_ERR_INTERN, "**unknowngpid", - "**unknowngpid %d %d", gpid[0], gpid[1] ); */ - return mpi_errno; - } - /* printf( "lpid[%d] = %d for gpid = (%d)%d\n", i, lpid[i], - gpid[0], gpid[1] ); */ - break; - } - } while (1); - gpid += 2; - } - } else { - for (i=0; i<size; i++) { - lpid[i] = *++gpid; gpid++; - } - return 0; - - } - - return mpi_errno; -} -/* - * The following routines convert to/from the global pids, which are - * represented as pairs of ints (process group id, rank in that process group) - */ - -/* FIXME: These routines belong in a different place */ -int MPID_GPID_GetAllInComm( MPIR_Comm *comm_ptr, int local_size, - int local_gpids[], int *singlePG ) -{ - int mpi_errno = MPI_SUCCESS; - int i; - int *gpid = local_gpids; - int lastPGID = -1, pgid; - MPID_VCR vc; - - MPIR_Assert(comm_ptr->local_size == local_size); - - if(mpidi_dynamic_tasking) { - *singlePG = 1; - for (i=0; i<comm_ptr->local_size; i++) { - vc = comm_ptr->vcr[i]; - - /* Get the process group id as an int */ - MPIDI_PG_IdToNum( vc->pg, &pgid ); - - *gpid++ = pgid; - if (lastPGID != pgid) { - if (lastPGID != -1) - *singlePG = 0; - lastPGID = pgid; - } - *gpid++ = vc->pg_rank; - - MPL_DBG_MSG_FMT(MPIR_DBG_COMM,VERBOSE, (MPL_DBG_FDEST, - "pgid=%d vc->pg_rank=%d", - pgid, vc->pg_rank)); - } - } else { - for (i=0; i<comm_ptr->local_size; i++) { - *gpid++ = 0; - (void)MPID_VCR_Get_lpid( comm_ptr->vcr[i], gpid ); - gpid++; - } - *singlePG = 1; - } - - return mpi_errno; -} - - -int MPIDI_VC_Init( MPID_VCR vcr, MPIDI_PG_t *pg, int rank ) -{ - vcr->pg = pg; - vcr->pg_rank = rank; -} - - -#endif diff --git a/src/mpid/pamid/src/mpidi_bufmm.c b/src/mpid/pamid/src/mpidi_bufmm.c deleted file mode 100644 index a466fe1..0000000 --- a/src/mpid/pamid/src/mpidi_bufmm.c +++ /dev/null @@ -1,716 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpidi_bufmm.c - * \brief Memory management for early arrivals - */ - - /*******************************************************************/ - /* DESCRIPTION: */ - /* Dynamic memory manager which allows allocation and */ - /* deallocation for earl arrivals sent via eager protocol. */ - /* */ - /* The basic method is for buffers of size between MIN_SIZE */ - /* and MAX_SIZE. The allocation scheme is a modified version */ - /* of Knuth's buddy algorithm. Regardless of what size buffer */ - /* is requested the size is rounded up to the nearest power of 2.*/ - /* Note that there is a buddy_header overhead per buffer (8 bytes*/ - /* in 32 bit and 16 bytes in 64 bit). So, for example, if a */ - /* 256 byte buffer is requested that would require 512 bytes */ - /* of memory. A 248-byte buffer needs 256 bytes of space. */ - /* Only for the maxsize buffers, it is guaranteed that the */ - /* allocation is a power of two, since typically applications */ - /* have such requirements. */ - /* */ - /* To speed up the buddy algorithm there are some preallocated */ - /* buffers. There are FLEX_NUM number of buffers from the */ - /* FLEX_COUNT number of smallest buffers. So, for example, if */ - /* MIN_SIZE is 16, FLEX_COUNT is 4, and FLEX_NUM is 256 then */ - /* there are buffers of size 16, 32, 64, and 128 preallocted */ - /* (256 buffers each). These buffers are arranged into stacks. */ - /* */ - /* If the system runs out of preallocated buffers or the size */ - /* is bigger than the biggest preallocated one then the buddy */ - /* algorithm is applied. Originally there is a list of MAX_SIZE */ - /* buffers. (The size is MAX_SIZE + the 8 or 16 byte overhead.) */ - /* These are not merged as the traditional buddy system would */ - /* require, since we never need bigger buffers than these. */ - /* Originally, the lists of smaller size buffers are empty. When */ - /* there is an allocation request the program searches for the */ - /* smallest free buffer available in the lists. If it is bigger */ - /* than the requested one then it is repeatedly split into half. */ - /* The other halves are inserted into the appropriate list of */ - /* free buffers. At deallocation the program attempts to merge */ - /* the buffer with it's buddy repeatedly to get the largest */ - /* buffer possible. */ - /*******************************************************************/ - -#include <mpidimpl.h> - -#define NO 0 -#define YES 1 -int application_set_eager_limit=0; - -#if TOKEN_FLOW_CONTROL -#define BUFFER_MEM_MAX (1<<26) /* 64MB */ -#define MAX_BUF_BKT_SIZE (1<<18) /* Max eager_limit is 256K */ -#define MIN_BUF_BKT_SIZE (64) -#define MIN_LOG2SIZE 6 /* minimum buffer size log-2 */ -#define MIN_SIZE 64 /* minimum buffer size */ -#define MAX_BUCKET 13 /* log of maximum buffer size/MIN_SIZE */ -#define MAX_SIZE (MIN_SIZE<<(MAX_BUCKET-1)) /* maximum buffer size */ -#define FLEX_COUNT 5 /* num of buf types to preallocate */ -#define FLEX_NUM 32 /* num of preallocated buffers */ - /* overhead for each buffer */ -#define MAX_BUDDIES 50 /* absolutely maximum number of buddies */ -#define BUDDY 1 -#define FLEX 0 -#define MAX_MALLOCS 10 - -#define OVERHEAD (sizeof(buddy_header) - 2 * sizeof (void *)) -#define TAB_SIZE (MAX_BUCKET+1) -#define ALIGN8(x) ((((unsigned long)(x)) + 0x7) & ~0x7L) -#ifndef max -#define max(a,b) ((a) > (b) ? (a) : (b)) -#endif -#ifndef min -#define min(a,b) (a>b ? b : a) -#endif - -/* normalize the size to number of MIN_SIZE blocks required to hold */ -/* required size. This includes the OVERHEAD */ -#define NORMSIZE(sz) (((sz)+MIN_SIZE+OVERHEAD-1) >> MIN_LOG2SIZE) - -typedef struct bhdr_struct{ - char buddy; /* buddy or flex alloc */ - char free; /* available or not */ - char bucket; /* log of buffer size */ - char *base_buddy; /* addr of max size buddy */ - struct bhdr_struct *next; /* list ptr, used if free */ - struct bhdr_struct *prev; /* list bptr, used if free */ -} buddy_header; - -typedef struct fhdr_struct{ - char buddy; /* buddy or flex alloc */ - char ind; /* which stack */ - /* bucket - min_bucket */ -} flex_header; - -typedef struct{ - void *ptr; /* malloc ptr need to be freed */ - int size; /* space allocted */ - int type; /* FLEX or BUDDY */ -} malloc_list_t; -malloc_list_t *malloc_list; - -static int nMallocs, maxMallocs; /* no. of malloc() issued */ -static int maxMallocs; /* max. no. of malloc() allowed */ -static int max_bucket; /* max. number of "buddy" bucket*/ -static int flex_count; /* pre-allocated bucket */ -static size_t max_size; /* max. size for each msg */ -static size_t flex_size; /* size for flex slot */ -static char *buddy_heap_ptr; /* ptr points to beg. of buddy */ -static char *end_heap_ptr; /* ptr points to end of buddy */ -static char *heap; /* begin. address of flex stack */ -static long mem_inuse; /* memory in use */ -long mem_hwmark; /* highest memory usage */ - - -static int sizetable[TAB_SIZE + 1]; /* from bucket to size */ -static int sizetrans[(1<<(MAX_BUCKET-1))+2];/* from size to power of 2 size*/ - -static char** flex_stack[TAB_SIZE]; /* for flex size alloc */ -static int flex_sp [TAB_SIZE]; /* flex stack pointer */ - -static buddy_header *free_buddy[TAB_SIZE];/* for buddy alloc */ - -int MPIDI_tfctrl_enabled=0; /* token flow control enabled */ -int MPIDI_tfctrl_hwmark=0; /* high water mark for tfc */ -int application_set_buf_mem=0; /* MP_BUFFER_MEM set by the user? */ -char *EagerLimit=NULL; /* export MP_EAGER_LIMIT if the */ - /* number is adjusted */ - -/***************************************************************************/ -/* calculate number of tokens available for each pair-wise communication. */ -/***************************************************************************/ - -void MPIDI_calc_tokens(int nTasks,uint *eager_limit_in, unsigned long *buf_mem_in ) -{ - char *cp; - unsigned long new_buf_mem_max,buf_mem_max; - int val; - int rc; - int i; - - /* Round up passed eager limit to power of 2 */ - if (MPIDI_Process.mp_buf_mem_max > *buf_mem_in) - buf_mem_max = MPIDI_Process.mp_buf_mem_max; - else - buf_mem_max= *buf_mem_in; - - if (*eager_limit_in != 0) { - for (val=1 ; val < *eager_limit_in ; val *= 2); - if (val > MAX_BUF_BKT_SIZE) { /* Maximum eager_limit is 256K */ - val = MAX_BUF_BKT_SIZE; - } - if (val < MIN_BUF_BKT_SIZE) { /* Minimum eager_limit is 64 */ - val = MIN_BUF_BKT_SIZE; - } - MPIDI_tfctrl_enabled = buf_mem_max / ((long)nTasks * val); - - /* We need to have a minimum of 2 tokens. If number of tokens is - * less than 2, re-calculate by reducing the eager-limit. - * If the number of tokens is still less than 2 then suggest a - * new minimum buf_mem. - */ - if (MPIDI_tfctrl_enabled < 2) { - for (val ; val>=MIN_BUF_BKT_SIZE; val/=2) { - MPIDI_tfctrl_enabled = (buf_mem_max) / ((long)nTasks * val); - if (MPIDI_tfctrl_enabled >= 2) { - break; - } - } - /* If the number of flow control tokens is still less than two */ - /* even with the eager_limit being reduced to 64, calculate a */ - /* new buf_mem value for 2 tokens and eager_limit = 64. */ - /* This will only happen if tasks>4K and original buf_mem=1M. */ - if (MPIDI_tfctrl_enabled < 2) { - /* Sometimes we are off by 1 - due to integer arithmetic. */ - new_buf_mem_max = (2 * nTasks * MIN_BUF_BKT_SIZE); - if ( (new_buf_mem_max <= BUFFER_MEM_MAX) && (!application_set_buf_mem) ) { - MPIDI_tfctrl_enabled = 2; - /* Reset val to mini (64) because the for loop above */ - /* would have changed it to 32. */ - val = MIN_BUF_BKT_SIZE; - buf_mem_max = new_buf_mem_max; - if ( application_set_buf_mem ) { - TRACE_ERR("informational messge \n"); - } - } - else { - /* Still not enough ...... Turn off eager send protocol */ - MPIDI_tfctrl_enabled = 0; - val = 0; - if (((MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0)|| (MPIDI_Process.mp_infolevel > 0)) - && (MPIR_Process.comm_world->rank == 0)) { - if ( application_set_buf_mem ) { - printf("ATTENTION: MP_BUFFER_MEM=%d was set too low, eager send protocol is disabled\n", - *buf_mem_in); - } - } - } - } - } - MPIDI_tfctrl_hwmark = (MPIDI_tfctrl_enabled+1) / 2; /* high water mark */ - /* Eager_limit may have been changed -- either rounded up or reduced */ - if ( *eager_limit_in != val ) { - if ( application_set_eager_limit && (*eager_limit_in > val)) { - /* Only give warning on reduce. */ - printf("ATTENTION: eager limit is reduced from %d to %d \n",*eager_limit_in,val); fflush(stdout); - } - *eager_limit_in = val; - - /* putenv MP_EAGER_LIMIT if it is changed */ - /* MP_EAGER_LIMIT always has a value. */ - /* need to check whether MP_EAGER_LIMIT has been export */ - EagerLimit = (char*)MPL_malloc(32 * sizeof(char)); - sprintf(EagerLimit, "MP_EAGER_LIMIT=%d",val); - rc = putenv(EagerLimit); - if (rc !=0) { - TRACE_ERR("PUTENV with Eager Limit failed \n"); - } - } - } - else { - /* Eager_limit = 0, all messages will be sent using rendezvous protocol */ - MPIDI_tfctrl_enabled = 0; - MPIDI_tfctrl_hwmark = 0; - } - /* user may want to set MP_EAGER_LIMIT to 0 or less than 256 */ - if (*eager_limit_in < MPIDI_Process.pt2pt.limits.application.immediate.remote) - MPIDI_Process.pt2pt.limits.application.immediate.remote= *eager_limit_in; - if (*eager_limit_in < MPIDI_Process.pt2pt.limits.application.eager.remote) - MPIDI_Process.pt2pt.limits.application.eager.remote= *eager_limit_in; -# ifdef DUMP_MM - printf("MPIDI_tfctrl_enabled=%d eager_limit=%d buf_mem=%d buf_mem_max=%d\n", - MPIDI_tfctrl_enabled,*eager_limit_in,*buf_mem_in,buf_mem_max); fflush(stdout); -# endif - -} - - - -/***************************************************************************/ -/* Initialize flex stack and stack pointers for each of FLEX_NUM slots */ -/***************************************************************************/ - -static char *MPIDI_init_flex(char *hp) -{ - int i,j,fcount; - char** area; - char *temp; - int size; - int kk; - - fcount = flex_count; - if ((fcount = flex_count) == 0) { - flex_size = 0; - return hp; - } -# ifdef DUMP_MM - printf("fcount=%d sizetable[fcount+1]=%d sizetable[1]=%d FLEX_NUM=%d overhead=%d\n", - fcount,sizetable[fcount+1],sizetable[1],FLEX_NUM,OVERHEAD); fflush(stdout); -# endif - flex_size = (sizetable[fcount+1] - sizetable[1]) *FLEX_NUM + - OVERHEAD * fcount * FLEX_NUM; - kk=fcount *FLEX_NUM *sizeof(char *); - size = ALIGN8(kk); - area = (char **) MPL_malloc(size); - malloc_list[nMallocs].ptr=(void *) area; - malloc_list[nMallocs].size=size; - malloc_list[nMallocs].type=FLEX; - nMallocs++; - - - flex_stack[0] = NULL; - for(i =1; i <=fcount; hp +=FLEX_NUM *(OVERHEAD +sizetable[i]), i++) { -# ifdef DEBUG - flex_heap[i] =hp; -# endif - flex_stack[i] =area; - area += FLEX_NUM; - flex_sp[i] =0; -# ifdef DUMP_MM - printf("MPIDI_init_flex() i=%d FLEX_NUM=%d fcount=%d i=%d flex_stack[i]=%p OVERHEAD=%d\n", - i,FLEX_NUM,fcount,i,flex_stack[i],OVERHEAD); fflush(stdout); -# endif - for(j =0; j <FLEX_NUM; j++){ - flex_stack[i][j] =hp +j *(OVERHEAD +sizetable[i]); -# ifdef DUMP_MM - printf("j=%d hp=%p advance =%d final=%x\n", j,(int *)hp,j *(OVERHEAD +sizetable[i]), - (int *) (hp +j *(OVERHEAD +sizetable[i]))); - fflush(stdout); -# endif - ((flex_header *)flex_stack[i][j])->buddy =FLEX; - ((flex_header *)flex_stack[i][j])->ind =i; -# ifdef DUMP_MM - printf("j=%d flex_stack[%02d][%02d] = %p advance=%d flag buddy=0x%x ind=0x%x\n", - j,i,j,((int *)(flex_stack[i][j])),(sizetable[i]+OVERHEAD), - ((flex_header *)flex_stack[i][j])->buddy, - ((flex_header *)flex_stack[i][j])->ind); fflush(stdout); -#endif - } - } - return hp; -} - -/***************************************************************************/ -/* Initialize buddy heap for each of MAX_BUDDIES slots */ -/***************************************************************************/ - - -static void MPIDI_alloc_buddies(int nums, int *space) -{ - int i; - uint size; - char *buddy,*prev; - - size = nums * (max_size +OVERHEAD); - buddy = buddy_heap_ptr; - if ((buddy_heap_ptr + size) > end_heap_ptr) { - /* preallocated space is exhausted, the caller needs to make */ - /* a malloc() call for storing the message */ - *space=NO; - return; - } - buddy_heap_ptr += size; - free_buddy[max_bucket] = (buddy_header *)buddy; - for(i =0, prev =NULL; i <nums; i++){ - ((buddy_header *)buddy)->buddy =BUDDY; - ((buddy_header *)buddy)->free =1; - ((buddy_header *)buddy)->bucket =max_bucket; - ((buddy_header *)buddy)->base_buddy =buddy; - ((buddy_header *)buddy)->prev =(buddy_header *)prev; - prev =buddy; -# ifdef DUMP_MM - printf("ALLOC_BUDDY i=%2d buddy=%d free=%d bucket=%d base_buddy=%p prev=%p next=%p max_size=%d \n", - i,(int)((buddy_header *)buddy)->buddy,(int)((buddy_header *)buddy)->free, - (int)((buddy_header *)buddy)->bucket,(int *) ((buddy_header *)buddy)->base_buddy, - (int *)((buddy_header *)buddy)->prev,(int *)(buddy + max_size +OVERHEAD),max_size); - fflush(stdout); -# endif - buddy +=max_size +OVERHEAD; - ((buddy_header *)prev)->next =(buddy_header *)buddy; - } - ((buddy_header *)prev)->next =NULL; -} - -/***************************************************************************/ -/* Initialize each of buddy slot */ -/***************************************************************************/ - -static void MPIDI_init_buddy(unsigned long buf_mem) -{ - int i; - int buddy_num; - size_t size; - int space=YES; - -# ifdef DEBUG - buddy_heap =buddy_heap_ptr; -# endif - for(i =0; i <= max_bucket; i++) - free_buddy[i] =NULL; - - /* figure out how many buddies we wanna preallocate - * size = BUFFER_MEM_SIZE >> 2; - * size -= flex_size; - */ - size = buf_mem; - size = size / (max_size + OVERHEAD); - size = (size == 0) ? 1 : (size > MAX_BUDDIES) ? MAX_BUDDIES : size; - MPIDI_alloc_buddies(size,&space); - if ( space == NO ) { - TRACE_ERR("out of memory %s(%d)\n",__FILE__,__LINE__); - MPID_abort(); - } -} - - -/***************************************************************************/ -/* initializ memory buffer for eager messages */ -/***************************************************************************/ - -int MPIDI_mm_init(int nTasks,uint *eager_limit_in,unsigned long *buf_mem_in) -{ - int i, bucket; - size_t size; - unsigned int eager_limit; - unsigned long buf_mem; - unsigned long buf_mem_max; - int need_allocation = 1; - - MPIDI_calc_tokens(nTasks,eager_limit_in, buf_mem_in); - buf_mem = *buf_mem_in; - eager_limit = *eager_limit_in; -# ifdef DEBUG - printf("Eager Limit=%d buf_mem=%ld tokens=%d hwmark=%d\n", - eager_limit,buf_mem, MPIDI_tfctrl_enabled,MPIDI_tfctrl_hwmark); - fflush(stdout); -# endif - if (eager_limit_in == 0) return 0; /* no EA buffer is needed */ - maxMallocs=MAX_MALLOCS; - malloc_list=(malloc_list_t *) MPL_malloc(maxMallocs * sizeof(malloc_list_t)); - if (malloc_list == NULL) return errno; - nMallocs=0; - mem_inuse=0; - mem_hwmark=0; - - - for (max_bucket=0,size=1 ; size < eager_limit ; max_bucket++,size *= 2); - max_size = 1 << max_bucket; - max_bucket -= (MIN_LOG2SIZE - 1); - flex_count = min(FLEX_COUNT,max_bucket); - for(i =1, size =MIN_SIZE; i <=MAX_BUCKET+1; i++, size =size << 1) - sizetable[i] =size; - sizetable[0] = 0; - - for (bucket=1, size = 1, i=1 ; bucket <= max_bucket; ) { - sizetrans[i++] = bucket; - if (i > size) { - size *= 2; - bucket++; - } - } - sizetrans[i] = sizetrans[i-1]; - /* 65536 is for flex stack which is not part of buf_mem_size */ - heap = MPL_malloc(buf_mem + 65536); - if (heap == NULL) return errno; - malloc_list[nMallocs].ptr=(void *) heap; - malloc_list[nMallocs].size=buf_mem + 65536; - malloc_list[nMallocs].type=BUDDY; - buddy_heap_ptr = heap; - end_heap_ptr = heap + buf_mem + 65536; -# ifdef DUMP_MM - printf("OVERHEAD=%d MAX_BUCKET=%d TAB_SIZE=%d buddy_header size=%d mem_size=%ld\n", - OVERHEAD,MAX_BUCKET,TAB_SIZE,sizeof(buddy_header),buf_mem); fflush(stdout); -# endif -# ifdef DEBUG - printf("nMallocs=%d ptr=%p size=%d type=%d bPtr=%p ePtr=%p\n", - nMallocs,(void *)malloc_list[nMallocs].ptr,malloc_list[nMallocs].size, - malloc_list[nMallocs].type,buddy_heap_ptr,end_heap_ptr); - fflush(stdout); -# endif - nMallocs++; - - buddy_heap_ptr = MPIDI_init_flex(heap); - MPIDI_init_buddy(buf_mem); -# ifdef MPIMM_DEBUG - if (mpimm_std_debug) { - printf("DevMemMgr uses\n"); - printf("\tmem-size=%d, maxbufsize=%ld\n",mem_size,max_size); - printf("\tfcount=%d fsize=%ld, max_bucket=%d\n", - flex_count,flex_size,max_bucket); - fflush(stdout); - } -# endif - return 0; -} - - -void MPIDI_close_mm() -{ - int i; - - if (nMallocs != 0) { - for (i=0; i< nMallocs; i++) { - MPL_free((void *) (malloc_list[i].ptr)); - } - MPL_free(malloc_list); - } -} - -/****************************************************************************/ -/* macros for MPIDI_mm_alloc() and MPIDI_mm_free() */ -/****************************************************************************/ - -#define MPIDI_flex_alloc(bucket) ((flex_sp[bucket] >=FLEX_NUM) ? \ - NULL : (char *)(flex_stack[bucket] \ - [flex_sp[bucket]++]) +OVERHEAD) - -#define MPIDI_flex_free(ptr) \ - int n; \ - ptr =(char *)ptr -OVERHEAD; \ - n =((flex_header *)ptr)->ind; \ - flex_stack[n][--flex_sp[n]] =(char *)ptr; - - -#define MPIDI_remove_head(ind) \ - { \ - if((free_buddy[ind] = \ - free_buddy[ind]->next) !=NULL) \ - free_buddy[ind]->prev =NULL; \ - } - -#define MPIDI_remove(bud) \ - { \ - if(bud->prev !=NULL) \ - bud->prev->next =bud->next; \ - else \ - free_buddy[bud->bucket] =bud->next; \ - if(bud->next !=NULL) \ - bud->next->prev =bud->prev; \ - } - -#define MPIDI_add_head(bud,ind) \ - { \ - if((bud->next =free_buddy[ind]) !=NULL)\ - free_buddy[ind]->prev =bud; \ - free_buddy[ind] =bud; \ - bud->prev =NULL; \ - } - -#define MPIDI_fill_header(bud,ind,base) \ - { \ - bud->buddy =BUDDY; \ - bud->free =1; \ - bud->bucket =ind; \ - bud->base_buddy =base; \ - } - -#define MPIDI_pair(bud,size) (((char*)(bud) - (bud)->base_buddy) & (size) ? \ - (buddy_header *)((char*)(bud) - (size)) : \ - (buddy_header *)((char*)(bud) + (size))) - -static buddy_header *MPIDI_split_buddy(int big,int bucket) -{ - buddy_header *bud,*buddy; - char *base; - int i; - - bud =free_buddy[big]; - MPIDI_remove_head(big); - base =bud->base_buddy; - for(i =big -1; i >=bucket; i--){ - buddy =(buddy_header *)((char*)bud +sizetable[i]); - MPIDI_fill_header(buddy,i,base); - MPIDI_add_head(buddy,i); - } - bud->bucket =bucket; - bud->free =0; - return bud; -} - -static void *MPIDI_buddy_alloc(int bucket) -{ - int i; - buddy_header *bud; - int space=YES; - -# ifdef TRACE - printf("(buddy) "); -# endif - if((bud =free_buddy[i =bucket]) ==NULL){ - i++; - do { - for(; i <=max_bucket; i++) - if(free_buddy[i] !=NULL){ - bud =MPIDI_split_buddy(i,bucket); - return (char *)bud +OVERHEAD; - } - MPIDI_alloc_buddies(1,&space); - if (space == NO) - return NULL; - i = max_bucket; - } while (1); - } - else{ - MPIDI_remove_head(bucket); - bud->free =0; - return (char *)bud +OVERHEAD; - } -} - -static buddy_header *MPIDI_merge_buddy(buddy_header *bud) -{ - buddy_header *buddy; - int size; - - while( (bud->bucket <max_bucket) && - (size = sizetable[bud->bucket]) && - ((buddy =MPIDI_pair(bud,size))->free) && - (buddy->bucket ==bud->bucket) ) - { - MPIDI_remove(buddy); - bud =min(bud,buddy); - bud->bucket++; - } - return bud; -} - -static void MPIDI_buddy_free(void *ptr) -{ - buddy_header *bud; - -# ifdef TRACE - printf("(buddy) "); -# endif - bud =(buddy_header *)((char *)ptr -OVERHEAD); - if(bud->bucket <max_bucket) - bud =MPIDI_merge_buddy(bud); - bud->free =1; - MPIDI_add_head(bud,bud->bucket); -} -# ifdef TRACE - int nAllocs =0; /* number of times MPIDI_mm_alloc() is called */ - int nFree =0; /* number of times MPIDI_mm_free() is called */ - int nM=0; /* number of times MPL_malloc() is called */ - int nF=0; /* number of times MPL_free() is called */ -# endif -void *MPIDI_mm_alloc(size_t size) -{ - void *pt; - int bucket,tmp; - - MPID_assert(size <= max_size); - tmp = NORMSIZE(size); - tmp =bucket =sizetrans[tmp]; - if(bucket >flex_count || (pt =MPIDI_flex_alloc(tmp)) ==NULL) { - pt =MPIDI_buddy_alloc(bucket); - if (MPIDI_Process.mp_statistics) { - mem_inuse = mem_inuse + sizetable[tmp]; - if (mem_inuse > mem_hwmark) { - mem_hwmark = mem_inuse; - } - } - } - if (pt == NULL) { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - pt=MPL_malloc(size); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - if (MPIDI_Process.mp_statistics) { - mem_inuse = mem_inuse + sizetable[tmp]; - if (mem_inuse > mem_hwmark) { - mem_hwmark = mem_inuse; - } - } -# ifdef TRACE - nM++; - if (pt == NULL) { - printf("ERROR line=%d\n",__LINE__); fflush(stdout); - } else { - printf("malloc nM=%d size=%d pt=0x%p \n",nM,size,pt); fflush(stdout); - } -# endif - } -# ifdef TRACE - printf("MPIDI_mm_alloc(%4d): %p\n",size,pt); - nAllocs++; -# endif - return pt; -} - -void MPIDI_mm_free(void *ptr, size_t size) -{ - int tmp,bucket; - - if (size > MAX_SIZE) { - TRACE_ERR("Out of memory in %s(%d)\n",__FILE__,__LINE__); - MPID_abort(); - } - if ((ptr >= (void *) heap) && (ptr < (void *)end_heap_ptr)) { - if(*((char *)ptr -OVERHEAD) ==FLEX){ - MPIDI_flex_free(ptr); - } - else - MPIDI_buddy_free(ptr); - if (MPIDI_Process.mp_statistics) { - tmp = NORMSIZE(size); - bucket =sizetrans[tmp]; - mem_inuse = mem_inuse - sizetable[bucket]; - if (mem_inuse > mem_hwmark) { - mem_hwmark = mem_inuse; - } - } - } else { - if (!ptr) { - TRACE_ERR("NULL ptr passed MPIDI_mm_free() in %s(%d)\n",__FILE__,__LINE__); - MPID_abort(); - } - if (MPIDI_Process.mp_statistics) { - tmp = NORMSIZE(size); - bucket =sizetrans[tmp]; - mem_inuse = mem_inuse - sizetable[bucket]; - if (mem_inuse > mem_hwmark) { - mem_hwmark = mem_inuse; - } - } - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); -# ifdef TRACE - nF++; - printf("free nF=%d size=%d ptr=0x%p \n",nF,sizetable[bucket],ptr); fflush(stdout); -# endif - MPL_free(ptr); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } -# ifdef TRACE - nFrees++; - printf("MPIDI_mm_free: %p \n",ptr); -# endif -} -#endif /* #if TOKEN_FLOW_CONTROL */ diff --git a/src/mpid/pamid/src/mpidi_env.c b/src/mpid/pamid/src/mpidi_env.c deleted file mode 100644 index 740bcee..0000000 --- a/src/mpid/pamid/src/mpidi_env.c +++ /dev/null @@ -1,1314 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpidi_env.c - * \brief Read Env Vars - */ - -#include <mpidimpl.h> - -#include "mpidi_util.h" - -/** \page env_vars Environment Variables - *************************************************************************** - * Pt2Pt Options * - *************************************************************************** - * - * - PAMID_EAGER - - * - PAMID_RZV - Sets the cutoff for the switch to the rendezvous protocol. - * The two options are identical. This takes an argument, in bytes, - * to switch from the eager protocol to the rendezvous protocol for point-to-point - * messaging. Increasing the limit might help for larger partitions - * and if most of the communication is nearest neighbor. - * - Default is 2049 bytes. - * - * - PAMID_EAGER_LOCAL - - * - PAMID_RZV_LOCAL - Sets the cutoff for the switch to the rendezvous protocol - * when the destination rank is local. The two options are identical. This - * takes an argument, in bytes, to switch from the eager protocol to the - * rendezvous protocol for point-to-point messaging. The default value - * effectively disables the eager protocol for local transfers. - * - Default is 0 bytes. - * - * - PAMID_RMA_PENDING - Maximum outstanding RMA requests. - * Limits number of PAMI_Request objects allocated by MPI Onesided operations. - * - Default is 1000. - * - * - PAMID_SHMEM_PT2PT - Determines if intranode point-to-point communication will - * use the optimized shared memory protocols. - * - Default is 1. - *************************************************************************** - * High-level Options * - *************************************************************************** - * - * - PAMID_THREAD_MULTIPLE - Specifies the messaging execution environment. - * It specifically selects whether there can be multiple independent - * communications occurring in parallel, driven by internal communications - * threads. - * - 0: The application threads drive the communications. No additional - * internal communications threads are used. - * - 1: There can be multiple independent communications occurring in - * parallel, driven by internal communications threads. - * Defaults to 0 when the application is linked with one of the "legacy" - * MPICH libraries (gcc.legacy, xl.legacy, xl.legacy.ndebug), or when - * MPI_Init_thread() is called without MPI_THREAD_MULTIPLE, and defaults - * to 1 when the application is linked with one of the "non-legacy" MPICH - * libraries (gcc, xl, xl.ndebug) and MPI_Init_thread() is called with - * MPI_THREAD_MULTIPLE. - * - NOTE: This environment variable has the same effect as setting - * MPIR_CVAR_DEFAULT_THREAD_LEVEL=multiple - * - * - PAMID_CONTEXT_MAX - This variable sets the maximum allowable number - * of contexts. Contexts are a method of dividing hardware resources - * among a parallel active messaging interface (PAMI) client (for example, - * MPI) to set how many parallel operations can occur at one time. Contexts - * are similar to channels in a communications system. The practical - * maximum is usually 64 processes per node. The default depends on the - * processes per node. - * - * - PAMID_CONTEXT_POST - This variable is required to take advantage of - * the parallelism of multiple contexts. This variable can increase - * latency, but it is the only way to allow parallelism among contexts. - * - 0: Only one parallel communications context can be used. Each - * operation runs in the application thread. - * - 1: Multiple parallel communications contexts can be used. An - * operation is posted to one of the contexts, and communications - * for that context are driven by communications threads. - * The default value is 1 when using the gcc, xl, and xl.ndebug libraries - * and MPI_Init_thread(... MPI_THREAD_MULTIPLE ...), and 0 otherwise. - * - * - PAMID_ASYNC_PROGRESS - This variable enables or disables the async - * progress extension. - * - * - PAMID_COLLECTIVES - Controls whether optimized collectives are used. - * Possible values: - * - 0 - Optimized collectives are not used. - * - 1 - Optimized collectives are used. - * If this is set to 0, only MPICH point-to-point based collectives will be used. - * - * - PAMID_COLLECTIVES_SELECTION - Turns on optimized collective selection. If this - * is not on, only the generic PGAS "always works" algorithms will be selected. This - * can still be better than PAMID_COLLECTIVES=0. Additionally, setting this off - * still allows environment variable selection of individual collectives protocols. - * - 0 - Optimized collective selection is not used. - * - 1 - Optimized collective selection is used. (default) - * - * - PAMID_COLLECTIVES_MEMORY_OPTIMIZED - Controls whether collectives are - * optimized to reduce memory usage. This may disable some PAMI collectives. - * Possible values: - * - 0 - Collectives are not memory optimized. - * - n - Collectives are memory optimized. Levels are bitwise values : - * MPIR_OPT_LVL_IRREG = 1, Do not optimize irregular communicators - * MPIR_OPT_LVL_NONCONTIG = 2, Disable some non-contig collectives - * - * PAMID_OPTIMIZED_SUBCOMMS - Use PAMI 'optimized' collectives. Defaullt is 1. - * - 0 - Some optimized protocols may be disabled. - * - 1 - All performance optimized protocols will be enabled when available - * - * - PAMID_VERBOSE - Increases the amount of information dumped during an - * MPI_Abort() call and during varoius MPI function calls. Possible values: - * - 0 - No additional information is dumped. - * - 1 - Useful information is printed from MPI_Init(). This option does - * NOT impact performance (other than a tiny bit during MPI_Init()) and - * is highly recommended for all applications. It will display which PAMID_ - * and BG_ env vars were used. However, it does NOT guarantee you typed the - * env vars in correctly :) - * - 2 - This turns on a lot of verbose output for collective selection, - * including a list of which protocols are viable for each communicator - * creation. This can be a lot of output, but typically only at - * communicator creation. Additionally, if PAMID_STATISTICS are on, - * queue depths for point-to-point operations will be printed for each node - * during MPI_Finalize(); - * - 3 - This turns on a lot of per-node information (plus everything - * at the verbose=2 level). This can be a lot of information and is - * rarely recommended. - * - Default is 0. However, setting to 1 is recommended. - * - * - PAMID_STATISTICS - Turns on statistics printing for the message layer - * such as the maximum receive queue depth. Possible values: - * - 0 - No statistics are printedcmf_bcas. - * - 1 - Statistics are printed. - * - Default is 0. - * - *************************************************************************** - *************************************************************************** - ** Collective Options ** - *************************************************************************** - *************************************************************************** - * - *************************************************************************** - * General Collectives Options * - *************************************************************************** - * - * - PAMID_NUMREQUESTS - Sets the number of outstanding asynchronous - * collectives to have before a barrier is called. This is used when - * the PAMI collective metadata indicates 'asyncflowctl' may be needed - * to avoid 'flooding' other participants with unexpected data. - * Higher numbers can help on larger partitions and larger message sizes. - * - * After every {PAMID_NUMREQUESTS} async collectives, the "glue" will call - * a barrier. - * - Default is 1 (guaranteed functionality) - * - N>1may used to tune performance - * - *************************************************************************** - * "Safety" Options * - *************************************************************************** - * - * - PAMID_SAFEALLGATHER - Some optimized allgather protocols require - * contiguous datatypes and similar datatypes on all nodes. To verify - * this is true, we must do an allreduce at the beginning of the - * allgather call. If the application uses "well behaved" datatypes, you can - * set this option to skip over the allreduce. This is most useful in - * irregular subcommunicators where the allreduce can be expensive and in - * applications calling MPI_Allgather() with simple/regular datatypes. - * Datatypes must also be 16-byte aligned to use the optimized - * broadcast-based allgather for larger allgather sendcounts. See - * PAMID_PREALLREDUCE for more options - * Possible values: - * - N - Perform the preallreduce. - * - Y - Skip the preallreduce. Setting this with "unsafe" datatypes will - * yield unpredictable results, usually hangs. - * - Default is N. - * - * - PAMID_SAFEALLGATHERV - The optimized allgatherv protocols require - * contiguous datatypes and similar datatypes on all nodes. Allgatherv - * also requires continuous displacements. To verify - * this is true, we must do an allreduce at the beginning of the - * allgatherv call. If the application uses "well behaved" datatypes and - * displacements, you can set this option to skip over the allreduce. - * This is most useful in irregular subcommunicators where the allreduce - * can be expensive and in applications calling MPI_Allgatherv() with - * simple/regular datatypes. - * Datatypes must also be 16-byte aligned to use the optimized - * broadcast-based allgather for larger allgather sendcounts. See - * PAMID_PREALLREDUCE for more options - * Possible values: - * - N - Perform the allreduce. - * - Y - Skip the allreduce. Setting this with "unsafe" datatypes will - * yield unpredictable results, usually hangs. - * - Default is N. - * - * - PAMID_SAFESCATTERV - The optimized scatterv protocol requires - * contiguous datatypes and similar datatypes on all nodes. It - * also requires continuous displacements. To verify - * this is true, we must do an allreduce at the beginning of the - * scatterv call. If the application uses "well behaved" datatypes and - * displacements, you can set this option to skip over the allreduce. - * This is most useful in irregular subcommunicators where the allreduce - * can be expensive. We have seen more applications with strange datatypes - * passed to scatterv than allgather/allgatherv/bcast/allreduce, so it is - * more likely you need to leave this at the default setting. However, we - * encourage you to try turning this off and seeing if your application - * completes successfully. - * Possible values: - * - N - Perform the allreduce. - * - Y - Skip the allreduce. Setting this with "unsafe" datatypes will - * yield unpredictable results, usually hangs. - * - Default is N. - * - * - PAMID_PREALLREDUCE - Controls the protocol used for the pre-allreducing - * employed by allgather(v), and scatterv. This option - * is independant from PAMID_ALLREDUCE. The available protocols can be - * determined with PAMID_VERBOSE=2. If collective selection is on, we will - * attempt to use the fastest protocol for the given communicator. - * - *************************************************************************** - * Specific Collectives Switches * - *************************************************************************** - * - * - PAMID_ALLGATHER - Controls the protocol used for allgather. - * Possible values: - * - MPICH - Turn off all optimizations for allgather and use the MPICH - * point-to-point protocol. This can be very bad on larger partitions. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_ALLGATHER=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_ALLGATHERV - Controls the protocol used for allgatherv. - * Possible values: - * - MPICH - Turn off all optimizations for allgather and use the MPICH - * point-to-point protocol. This can be very bad on larger partitions. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_ALLGATHERV=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_ALLREDUCE - Controls the protocol used for allreduce. - * Possible values: - * - MPICH - Turn off all optimizations for allreduce and use the MPICH - * point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * Note: Many allreduce algorithms are in the "must query" category and - * might or might not be available for a specific callsite. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_ALLREDUCE=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_ALLTOALL - - * - PAMID_ALLTOALLV - Controls the protocol used for - * alltoall/alltoallv Possible values: - * - MPICH - Turn off all optimizations and use the MPICH - * point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_ALLTOALL=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - * - PAMID_BARRIER - Controls the protocol used for barriers. - * Possible values: - * - MPICH - Turn off optimized barriers and use the MPICH - * point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_BARRIER=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_BCAST - Controls the protocol used for broadcast. Possible values: - * - MPICH - Turn off all optimizations for broadcast and use the MPICH - * point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_BCAST=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_GATHER - Controls the protocol used for gather. - * Possible values: - * - MPICH - Use the MPICH point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_GATHER=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_REDUCE - Controls the protocol used for reduce. - * Possible values: - * - MPICH - Turn off all optimizations and use the MPICH point-to-point - * protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_REDUCE=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_REDUCE_SCATTER - Controls the protocol used for reduce_scatter - * operations. The options for PAMID_SCATTERV and PAMID_REDUCE can change the - * behavior of reduce_scatter. Possible values: - * - MPICH - Use the MPICH point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_REDUCESCATTER=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_SCATTER - Controls the protocol used for scatter. - * Possible values: - * - MPICH - Use the MPICH point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_SCATTER=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - PAMID_SCATTERV - Controls the protocol used for scatterv. - * Possible values: - * - MPICH - Use the MPICH point-to-point protocol. - * - Other options can be determined by running with PAMID_VERBOSE=2. - * TODO: Implement the NO options - * NOTE: Individual options may be turned off by prefixing them with "NO". - * For example, PAMID_SCATTERV=NO{specific protocol name} will prevent the - * use of the specified protocol. - * - * - */ -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) -int prtStat=0; -int prtEnv=0; - -MPIX_stats_t *mpid_statp=NULL; -extern MPIDI_printenv_t *mpich_env; -#endif - -#define ENV_Deprecated(a, b, c, d, e) ENV_Deprecated__(a, b, c, d, e) - -#ifdef TOKEN_FLOW_CONTORL - extern int MPIDI_get_buf_mem(unsigned long *); - extern int MPIDI_atoi(char* , int* ); -#endif - extern int application_set_eager_limit; - -static inline void -ENV_Deprecated__(char* name[], unsigned num_supported, unsigned* deprecated, int rank, int NA) -{ - if (name == NULL) return; - - unsigned i; - char * env; - - for (i=0; i<num_supported; ++i) - { - if (name[i] == NULL) return; - - if (NA) - { - env = getenv(name[i]); - if (env != NULL) - { - if (rank == 0) - { - if (*deprecated == 0) - fprintf (stderr, "\n"); - - fprintf (stderr, "The environment variable \"%s\" is not applicable.\n", name[i]); - } - *deprecated = 1; - } - } - } - - for (i=num_supported; name[i] != NULL; ++i) - { - env = getenv(name[i]); - if (env != NULL) - { - if (rank == 0) - { - if (*deprecated == 0) - fprintf (stderr, "\n"); - - if (NA) - fprintf (stderr, "The environment variable \"%s\" is deprecated.\n", name[i]); - else - { - char supported[10240]; - int n, index = 0; - index += snprintf(&supported[index], 10240-index-1, "\"%s\"", name[0]); - for (n=1; n<num_supported; ++n) - index += snprintf(&supported[index], 10240-index-1, " or \"%s\"", name[n]); - - fprintf (stderr, "The environment variable \"%s\" is deprecated. Consider using %s instead.\n", name[i], supported); - } - } - *deprecated = 1; - } - } -} - -#define ENV_Unsigned(a, b, c, d, e) ENV_Unsigned__(a, b, #b, c, d, e, 0) - -#define ENV_Unsigned_NA(a, b, c, d, e) ENV_Unsigned__(a, b, #b, c, d, e, 1) - -static inline void -ENV_Unsigned__(char* name[], unsigned* val, char* string, unsigned num_supported, unsigned* deprecated, int rank, int NA) -{ - /* Check for deprecated environment variables. */ - if (deprecated != NULL) - ENV_Deprecated(name, num_supported, deprecated, rank, NA); - - char * env; - int rc; - - unsigned i=0; - for (;; ++i) { - if (name[i] == NULL) - return; - env = getenv(name[i]); - if (env != NULL) - break; - } - - unsigned oldval = *val; - rc=MPIDI_atoi(env,val); - if(rc != 0) - { - /* Something went wrong with the processing this integer - * Print a warning, and restore the original value */ - *val = oldval; - fprintf(stderr, "Warning: Environment variable: %s should be an integer value: defaulting to %d", string, *val); - return; - } - - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL && rank == 0) - fprintf(stderr, "%s = %u\n", string, *val); -} - - -#define ENV_Char(a, b) ENV_Char__(a, b, #b) -static inline void -ENV_Char__(char* name[], unsigned* val, char* string) -{ - char * env; - unsigned i=0; - for (;; ++i) { - if (name[i] == NULL) - return; - env = getenv(name[i]); - if (env != NULL) - break; - } - - if ( (env[0]=='y')|| (env[0]=='Y') - || (env[0]=='p')|| (env[0]=='P') - || (env[0]=='t')|| (env[0]=='T') - || (env[0]=='r')|| (env[0]=='R') - || (env[0]=='F')|| (env[0]=='f')) - *val = 1; - /*This may seem redundant; however, - in some cases we need to force val=0 if value = no/none*/ - if ((env[0]=='n')|| (env[0]=='N')) - *val = 0; - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) - fprintf(stderr, "%s = %u\n", string, *val); -} - -#define ENV_Char2(a, b) ENV_Char2__(a, b, #b) -static inline void -ENV_Char2__(char* name[], char** val, char* string) -{ - unsigned i=0; - for (;; ++i) { - if (name[i] == NULL) - return; - *val = getenv(name[i]); - if (*val != NULL) - break; - } - - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_DETAILS_ALL) - fprintf(stderr, "%s = %s\n", string, *val); -} - -/** \brief Checks the Environment variables at initialization and stores the results. - * - * \param [in] rank The process rank; used to limit output - * \param [in] requested The thread model requested by the user - */ -void -MPIDI_Env_setup(int rank, int requested) -{ - unsigned found_deprecated_env_var = 0; - - /* Set defaults for various environment variables */ - - /* Set the verbosity level. When set, this will print information at finalize. */ - { - char* names[] = {"PAMID_VERBOSE", "PAMI_VERBOSE", NULL}; - ENV_Unsigned(names, &MPIDI_Process.verbose, 1, &found_deprecated_env_var, rank); - } - - /* Enable statistics collection. */ - { - char* names[] = {"PAMID_STATISTICS", "PAMI_STATISTICS", NULL}; - ENV_Unsigned(names, &MPIDI_Process.statistics, 1, &found_deprecated_env_var, rank); - } - - /* Set async flow control - number of collectives between barriers */ - { - char* names[] = {"PAMID_NUMREQUESTS", NULL}; - ENV_Unsigned(names, &MPIDI_Process.optimized.num_requests, 1, &found_deprecated_env_var, rank); - TRACE_ERR("MPIDI_Process.optimized.num_requests=%u\n", MPIDI_Process.optimized.num_requests); - } - - /* "Globally" set the optimization flag for low-level collectives in geometry creation. - * This is probably temporary. metadata should set this flag likely. - */ - { - /* THIS ENVIRONMENT VARIABLE NEEDS TO BE DOCUMENTED ABOVE */ - char *names[] = {"PAMID_OPTIMIZED_SUBCOMMS", "PAMI_OPTIMIZED_SUBCOMMS", NULL}; - ENV_Unsigned(names, &MPIDI_Process.optimized.subcomms, 1, &found_deprecated_env_var, rank); - TRACE_ERR("MPIDI_Process.optimized.subcomms=%u\n", MPIDI_Process.optimized.subcomms); - } - - /* Set the threads values from a single override. */ - { - unsigned value = (unsigned)-1; - char* names[] = {"PAMID_THREAD_MULTIPLE", NULL}; - ENV_Unsigned(names, &value, 1, &found_deprecated_env_var, rank); -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - /* Any mpich work function posted to a context that eventually initiates - * other communcation transfers will hang on a lock attempt if the - * 'context post' feature is not enabled. Until this code flow is fixed - * the context post must not be disabled. - * - * See discussion in src/mpid/pamid/include/mpidi_macros.h - * -> MPIDI_Context_post() - */ - MPIDI_Process.perobj.context_post.requested = 1; - if (value == 1) /* force on */ - { - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_DEFAULT; - MPIDI_Process.avail_contexts = MPIDI_MAX_CONTEXTS; - } - else if (value == 0) /* force off */ - { - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_DISABLED; - MPIDI_Process.avail_contexts = 1; - } - else if (requested != MPI_THREAD_MULTIPLE) - { -#ifdef BGQ_SUPPORTS_TRIGGER_ASYNC_PROGRESS - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_TRIGGER; - MPIDI_Process.avail_contexts = MPIDI_MAX_CONTEXTS; -#else - /* BGQ does not support the 'trigger' style of async progress. Until - * this is implemented, set the async progress configuration to the - * 'single threaded' defaults. - */ - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_DISABLED; - MPIDI_Process.avail_contexts = 1; -#endif - } -#else - /* The only valid async progress mode when using the 'global' mpich lock - * mode is the 'trigger' async progress mode. Also, the 'global' mpich lock - * mode only supports a single context. - * - * See discussions in mpich/src/mpid/pamid/src/mpid_init.c and - * src/mpid/pamid/src/mpid_progress.h for more information. - */ - if (value == 1) /* force on */ - { - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_TRIGGER; - MPIDI_Process.avail_contexts = 1; - } - else if (value == 0) /* force off */ - { - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_DISABLED; - MPIDI_Process.avail_contexts = 1; - } -#endif - } - - /* Set the upper-limit of number of PAMI Contexts. */ - { - unsigned value = (unsigned)-1; - char *names[] = {"PAMID_CONTEXT_MAX", "PAMI_MAXCONTEXT", "PAMI_MAXCONTEXTS", "PAMI_MAX_CONTEXT", "PAMI_MAX_CONTEXTS", NULL}; - ENV_Unsigned(names, &value, 1, &found_deprecated_env_var, rank); - - if (value != -1) - { -#if (MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ) - /* The 'global' mpich lock mode only supports a single context. - * See discussion in mpich/src/mpid/pamid/src/mpid_init.c for more - * information. - */ - if (value > 1) - { - found_deprecated_env_var++; - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0 && rank == 0) - fprintf(stderr, "The environment variable \"PAMID_CONTEXT_MAX\" is invalid as this mpich library was configured.\n"); - } -#endif - if (value == 0) - { - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0 && rank == 0) - fprintf(stderr, "The environment variable \"PAMID_CONTEXT_MAX\" must specify a value > 0. The number of contexts will be set to 1\n"); - - value=1; - } - - MPIDI_Process.avail_contexts=value; - } - - TRACE_ERR("MPIDI_Process.avail_contexts=%u\n", MPIDI_Process.avail_contexts); - } - - /* Enable context post. */ - { - unsigned value = (unsigned)-1; - char *names[] = {"PAMID_CONTEXT_POST", "PAMI_CONTEXT_POST", NULL}; - ENV_Unsigned(names, &value, 1, &found_deprecated_env_var, rank); - if (value != -1) - { -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - MPIDI_Process.perobj.context_post.requested = (value > 0); -#else - found_deprecated_env_var++; - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0 && rank == 0) - fprintf(stderr, "The environment variable \"PAMID_CONTEXT_POST\" is invalid as this mpich library was configured.\n"); -#endif - } - TRACE_ERR("MPIDI_Process.perobj.context_post.requested=%u\n", MPIDI_Process.perobj.context_post.requested); - } - - /* Enable/Disable asynchronous progress. */ - { - /* THIS ENVIRONMENT VARIABLE NEEDS TO BE DOCUMENTED ABOVE */ - unsigned value = (unsigned)-1; - char *names[] = {"PAMID_ASYNC_PROGRESS", "PAMID_COMMTHREADS", "PAMI_COMMTHREAD", "PAMI_COMMTHREADS", "PAMI_COMM_THREAD", "PAMI_COMM_THREADS", NULL}; - ENV_Unsigned(names, &value, 1, &found_deprecated_env_var, rank); - - if (value != (unsigned)-1) - { - if (value != ASYNC_PROGRESS_MODE_DISABLED) - { -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - if (value == ASYNC_PROGRESS_MODE_LOCKED && - MPIDI_Process.perobj.context_post.requested == 0) - { - /* The 'locked' async progress mode requires context post. - * - * See discussion in src/mpid/pamid/src/mpid_progress.h for more - * information. - */ - found_deprecated_env_var++; - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0 && rank == 0) - fprintf(stderr, "The environment variable \"PAMID_ASYNC_PROGRESS=1\" requires \"PAMID_CONTEXT_POST=1\".\n"); - } - -#else /* (MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ) */ - if (value == ASYNC_PROGRESS_MODE_LOCKED) - { - /* The only valid async progress mode when using the 'global' mpich - * lock mode is the 'trigger' async progress mode. - * - * See discussion in src/mpid/pamid/src/mpid_progress.h for more - * information. - */ - found_deprecated_env_var++; - if (MPIDI_Process.verbose >= MPIDI_VERBOSE_SUMMARY_0 && rank == 0) - fprintf(stderr, "The environment variable \"PAMID_ASYNC_PROGRESS=1\" is invalid as this mpich library was configured.\n"); - - } -#endif - } - - MPIDI_Process.async_progress.mode = value; - } - TRACE_ERR("MPIDI_Process.async_progress.mode=%u\n", MPIDI_Process.async_progress.mode); - } - - /* - * Determine 'short' limit - * - sets both the 'local' and 'remote' short limit, and - * - sets both the 'application' and 'internal' short limit - * - * Identical to setting the PAMID_PT2PT_LIMITS environment variable as: - * - * PAMID_PT2PT_LIMITS="::x:x:::x:x" - */ - { - /* THIS ENVIRONMENT VARIABLE NEEDS TO BE DOCUMENTED ABOVE */ - char* names[] = {"PAMID_SHORT", "MP_S_SHORT_LIMIT", "PAMI_SHORT", NULL}; - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.application.immediate.remote, 2, &found_deprecated_env_var, rank); - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.application.immediate.local, 2, NULL, rank); - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.internal.immediate.remote, 2, NULL, rank); - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.internal.immediate.local, 2, NULL, rank); - } - - /* - * Determine 'remote' eager limit - * - sets both the 'application' and 'internal' remote eager limit - * - * Identical to setting the PAMID_PT2PT_LIMITS environment variable as: - * - * PAMID_PT2PT_LIMITS="x::::x:::" - * -- or -- - * PAMID_PT2PT_LIMITS="x::::x" - */ - { - char* names[] = {"PAMID_EAGER", "PAMID_RZV", "MP_EAGER_LIMIT", "PAMI_RVZ", "PAMI_RZV", "PAMI_EAGER", NULL}; - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.application.eager.remote, 3, &found_deprecated_env_var, rank); - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.internal.eager.remote, 3, NULL, rank); - } -#if TOKEN_FLOW_CONTROL - /* Determine if users set eager limit */ - { - MPIDI_set_eager_limit(&MPIDI_Process.pt2pt.limits.application.eager.remote); - } - /* Determine buffer memory for early arrivals */ - { - int rc; - rc=MPIDI_get_buf_mem(&MPIDI_Process.mp_buf_mem,&MPIDI_Process.mp_buf_mem_max); - MPID_assert_always(rc == MPI_SUCCESS); - } -#endif - - /* - * Determine 'local' eager limit - * - sets both the 'application' and 'internal' local eager limit - * - * Identical to setting the PAMID_PT2PT_LIMITS environment variable as: - * - * PAMID_PT2PT_LIMITS=":x::::x::" - * -- or -- - * PAMID_PT2PT_LIMITS=":x::::x" - */ - { - char* names[] = {"PAMID_RZV_LOCAL", "PAMID_EAGER_LOCAL", "MP_EAGER_LIMIT_LOCAL", "PAMI_RVZ_LOCAL", "PAMI_RZV_LOCAL", "PAMI_EAGER_LOCAL", NULL}; - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.application.eager.local, 3, &found_deprecated_env_var, rank); - ENV_Unsigned(names, &MPIDI_Process.pt2pt.limits.internal.eager.local, 3, NULL, rank); - } - - /* - * Determine *all* point-to-point limit overrides. - * - * The entire point-to-point limit set is determined by three boolean - * configuration values: - * - 'is non-local limit' vs 'is local limit' - * - 'is eager limit' vs 'is immediate limit' - * - 'is application limit' vs 'is internal limit' - * - * The point-to-point configuration limit values are specified in order and - * are delimited by ':' characters. If a value is not specified for a given - * configuration then the limit is not changed. All eight configuration - * values are not required to be specified, although in order to set the - * last (eighth) configuration value the previous seven configurations must - * be listed. For example: - * - * PAMID_PT2PT_LIMITS=":::::::10240" - * - * The configuration entries can be described as: - * 0 - remote eager application limit - * 1 - local eager application limit - * 2 - remote immediate application limit - * 3 - local immediate application limit - * 4 - remote eager internal limit - * 5 - local eager internal limit - * 6 - remote immediate internal limit - * 7 - local immediate internal limit - * - * Examples: - * - * "10240" - * - sets the application internode eager (the "normal" eager limit) - * - * "10240::64" - * - sets the application internode eager and immediate limits - * - * "::::0:0:0:0" - * - disables 'eager' and 'immediate' for all internal point-to-point - */ - { - char * env = getenv("PAMID_PT2PT_LIMITS"); - if (env != NULL) - { - size_t i, n = strlen(env); - char * tmp = (char *) MPL_malloc(n+1); - strncpy(tmp,env,n); - if (n>0) tmp[n]=0; - - char * tail = tmp; - char * token = tail; - for (i = 0; token == tail; i++) - { - while (*tail != 0 && *tail != ':') tail++; - if (*tail == ':') - { - *tail = 0; - if (token != tail) - MPIDI_atoi(token, &MPIDI_Process.pt2pt.limits_array[i]); - tail++; - token = tail; - } - else - { - if (token != tail) - MPIDI_atoi(token, &MPIDI_Process.pt2pt.limits_array[i]); - } - } - - MPL_free (tmp); - } - } - - /* Set the maximum number of outstanding RDMA requests */ - { - char* names[] = {"PAMID_RMA_PENDING", "MP_RMA_PENDING", "PAMI_RMA_PENDING", NULL}; - ENV_Unsigned(names, &MPIDI_Process.rma_pending, 2, &found_deprecated_env_var, rank); - } - - /* Set the status of the optimized collectives */ - { - char* names[] = {"PAMID_COLLECTIVES", "PAMI_COLLECTIVE", "PAMI_COLLECTIVES", NULL}; - ENV_Unsigned(names, &MPIDI_Process.optimized.collectives, 1, &found_deprecated_env_var, rank); - TRACE_ERR("MPIDI_Process.optimized.collectives=%u\n", MPIDI_Process.optimized.collectives); - } - - /* First, if MP_COLLECTIVE_OFFLOAD is "on", then we want PE (FCA) collectives */ - { - unsigned temp; - temp = 0; - char* names[] = {"MP_COLLECTIVE_OFFLOAD", NULL}; - ENV_Char(names, &temp); - if(temp) - MPIDI_Process.optimized.collectives = MPID_COLL_FCA; - } - - /* However, MP_MP_PAMI_FOR can be set to "none" in which case we don't want PE (FCA) collectives */ - { - char *env = getenv("MP_MPI_PAMI_FOR"); - if(env != NULL) - { - if(strncasecmp(env, "N", 1) == 1) - MPIDI_Process.optimized.collectives = MPID_COLL_OFF; - } - } - - /* Set the status for optimized selection of collectives */ - { - char* names[] = {"PAMID_COLLECTIVES_SELECTION", NULL}; - ENV_Unsigned(names, &MPIDI_Process.optimized.select_colls, 1, &found_deprecated_env_var, rank); - TRACE_ERR("MPIDI_Process.optimized.select_colls=%u\n", MPIDI_Process.optimized.select_colls); - } - - /* Finally, if MP_COLLECTIVE_SELECTION is "on", then we want to overwrite any other setting */ - { - char *env = getenv("MP_COLLECTIVE_SELECTION"); - if(env != NULL) - { - pami_extension_t extension; - pami_result_t status = PAMI_ERROR; - status = PAMI_Extension_open (MPIDI_Client, "EXT_collsel", &extension); - if(status == PAMI_SUCCESS) - { - char *env = getenv("MP_COLLECTIVE_SELECTION"); - if(env != NULL) - { - if(strncasecmp(env, "TUN", 3) == 0) - { - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_TUNE; - if(MPIDI_Process.optimized.collectives != MPID_COLL_FCA) - MPIDI_Process.optimized.collectives = MPID_COLL_ON; - } - else if(strncasecmp(env, "YES", 3) == 0) - { - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_ALL; /* All collectives will be using auto coll sel. - We will check later on each individual coll. */ - if(MPIDI_Process.optimized.collectives != MPID_COLL_FCA) - MPIDI_Process.optimized.collectives = MPID_COLL_ON; - } - else - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE; - - } - } - else - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE;/* Auto coll sel is disabled for all */ - } - else - MPIDI_Process.optimized.auto_select_colls = MPID_AUTO_SELECT_COLLS_NONE;/* Auto coll sel is disabled for all */ - } - - /* Set the status for memory optimized collectives */ - { - char* names[] = {"PAMID_COLLECTIVES_MEMORY_OPTIMIZED", NULL}; - ENV_Unsigned(names, &MPIDI_Process.optimized.memory, 1, &found_deprecated_env_var, rank); - TRACE_ERR("MPIDI_Process.optimized.memory=%u\n", MPIDI_Process.optimized.memory); - } - - /* Set the status of the optimized shared memory point-to-point functions */ - { - char* names[] = {"PAMID_SHMEM_PT2PT", "PAMI_SHMEM_PT2PT", NULL}; - ENV_Unsigned(names, &MPIDI_Process.shmem_pt2pt, 2, &found_deprecated_env_var, rank); - } - - /* MP_SHMEM_PT2PT = yes or no */ - { - char* names[] = {"MP_SHMEM_PT2PT", NULL}; - ENV_Char(names, &MPIDI_Process.shmem_pt2pt); - } - - /* Enable MPIR_* implementations of non-blocking collectives */ - { - char* names[] = {"PAMID_MPIR_NBC", NULL}; - ENV_Unsigned(names, &MPIDI_Process.mpir_nbc, 1, &found_deprecated_env_var, rank); - } - - /* Enable typed PAMI calls for derived types within MPID_Put and MPID_Get. */ - { - char* names[] = {"PAMID_TYPED_ONESIDED", NULL}; - ENV_Unsigned(names, &MPIDI_Process.typed_onesided, 1, &found_deprecated_env_var, rank); - } - /* Check for deprecated collectives environment variables. These variables are - * used in src/mpid/pamid/src/comm/mpid_selectcolls.c */ - { - { - char* names[] = {"PAMID_COLLECTIVE_ALLGATHER", "PAMI_ALLGATHER", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLGATHER_PREALLREDUCE", "PAMI_ALLGATHER_PREALLREDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLGATHERV", "PAMI_ALLGATHERV", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLGATHERV_PREALLREDUCE", "PAMI_ALLGATHERV_PREALLREDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLREDUCE", "PAMI_ALLREDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLREDUCE_PREALLREDUCE", "PAMI_ALLREDUCE_PREALLREDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLTOALL", "PAMI_ALLTOALL", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLTOALLV", "PAMI_ALLTOALLV", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_ALLTOALLV_INT", "PAMI_ALLTOALLV_INT", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_BARRIER", "PAMI_BARRIER", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_BCAST", "PAMI_BCAST", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_BCAST_PREALLREDUCE", "PAMI_BCAST_PREALLREDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_GATHER", "PAMI_GATHER", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_GATHERV", "PAMI_GATHERV", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_REDUCE", "PAMI_REDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_SCAN", "PAMI_SCAN", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_SCATTER", "PAMI_SCATTER", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_SCATTERV", "PAMI_SCATTERV", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_COLLECTIVE_SCATTERV_PREALLREDUCE", "PAMI_SCATTERV_PREALLREDUCE", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } - { - char* names[] = {"PAMID_CORE_ON_ABORT", "PAMI_COREONABORT", "PAMI_COREONMPIABORT", "PAMI_COREONMPIDABORT", NULL}; - ENV_Deprecated(names, 1, &found_deprecated_env_var, rank, 0); - } -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - mpich_env=(MPIDI_printenv_t *) MPL_malloc(sizeof(MPIDI_printenv_t)+1); - mpid_statp=(MPIX_stats_t *) MPL_malloc(sizeof(MPIX_stats_t)+1); - memset((void *)mpich_env,0,sizeof(MPIDI_printenv_t)); - memset((void *)mpid_statp,0,sizeof(MPIX_stats_t)); - /* If MP_STATISTICS is set, each task prints statistics data at the end of an MPI jobs */ - { - char* names[] = {"MP_STATISTICS", NULL}; - ENV_Char(names, &MPIDI_Process.mp_statistics); - if (MPIDI_Process.mp_statistics) { - prtStat=1; - } - } - /* if MP_INFOLEVEL is greater than or equal to 1, the library will display the banner */ - { - char* names[] = {"MP_INFOLEVEL", NULL}; - ENV_Unsigned(names, &MPIDI_Process.mp_infolevel, 1, &found_deprecated_env_var, rank); - } - /* If MP_PRINTENV is set, task 0 prints out the env. info after MPI_Init */ - { - char* names[] = {"MP_PRINTENV", NULL}; - ENV_Char(names, &MPIDI_Process.mp_printenv); - if (MPIDI_Process.mp_printenv) { - prtEnv=1; - } - } - /* MP_CSS_INTERRUPT */ - { - char *cp=NULL, *cp1=NULL; - int user_interrupts=0; - char* names[] = {"MP_CSS_INTERRUPT", NULL}; - ENV_Char(names, &mpich_env->interrupts); - if (mpich_env->interrupts == 1) /* force on */ - { - cp = getenv("MP_CSS_INTERRUPT"); - if (*cp=='Y' || *cp=='y') - { - user_interrupts = ASYNC_PROGRESS_ALL; - } - else - { - char delimiter='+'; - cp1 = strchr(cp,delimiter); - if (!cp1) /* timer or receive */ - { - if ( (*cp == 't') || (*cp == 'T') ) - user_interrupts = PAMIX_PROGRESS_TIMER; - else if ( (*cp == 'r') || (*cp == 'R') ) - user_interrupts = PAMIX_PROGRESS_RECV_INTERRUPT; - } - else /* timer + receive */ - { - if ((( *cp == 't' || *cp == 'T') && ( *(cp1+1) == 'r' || *(cp1+1) == 'R')) || - (( *cp == 'r' || *cp == 'R') && ( *(cp1+1) == 't' || *(cp1+1) == 'T'))) - user_interrupts = ASYNC_PROGRESS_ALL; - else - { - TRACE_ERR("ERROR in MP_CSS_INTERRUPT %s(%d)\n",__FILE__,__LINE__); - exit(1); - } - } - } - MPIDI_Process.mp_interrupts=user_interrupts; - MPIDI_Process.perobj.context_post.requested = 0; - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_TRIGGER; -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - MPIDI_Process.avail_contexts = MPIDI_MAX_CONTEXTS; -#else - MPIDI_Process.avail_contexts = 1; -#endif - } - else if (mpich_env->interrupts == 0) /* force off */ - { - MPIDI_Process.perobj.context_post.requested = 0; - MPIDI_Process.async_progress.mode = ASYNC_PROGRESS_MODE_DISABLED; - MPIDI_Process.avail_contexts = 1; - } - } - /* MP_POLLING_INTERVAL */ - { - char* names[] = {"MP_POLLING_INTERVAL", NULL}; - ENV_Unsigned(names,(unsigned int *) &mpich_env->polling_interval,1, &found_deprecated_env_var, rank); - } - - /* MP_RETRANSMIT_INTERVAL */ - { - char* names[] = {"MP_RETRANSMIT_INTERVAL", NULL}; - ENV_Unsigned(names, &mpich_env->retransmit_interval,1, &found_deprecated_env_var, rank); - } - - /* MP_S_USE_PAMI_GET */ - { - char* names[] = {"MP_S_USE_PAMI_GET", NULL}; - ENV_Char(names, &MPIDI_Process.mp_s_use_pami_get); - } -#endif - /* MP_S_SMP_DETECT */ - { - char* names[] = {"MP_S_SMP_DETECT", "PAMID_SMP_DIRECT", NULL}; - ENV_Char(names, &MPIDI_Process.smp_detect); - if(!MPIDI_Process.smp_detect) - PAMIX_Extensions.is_local_task.node_info=NULL; - } - - } - { -#if TOKEN_FLOW_CONTROL - char* names[] = {"MP_USE_TOKEN_FLOW_CONTROL", NULL}; - ENV_Char(names, &MPIDI_Process.is_token_flow_control_on); - if (!MPIDI_Process.is_token_flow_control_on) - MPIDI_Process.mp_buf_mem=0; -#endif - } - -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - char* names[] = {"MP_S_USE_QUEUE_BINARY_SEARCH_SUPPORT", NULL}; - ENV_Char(names, &MPIDI_Process.queue_binary_search_support_on); -#endif - -#if CUDA_AWARE_SUPPORT - char* names[] = {"MP_CUDA_AWARE", NULL}; - ENV_Char(names, &MPIDI_Process.cuda_aware_support_on); - if(MPIDI_Process.cuda_aware_support_on && MPIDI_enable_cuda() == false) - { - MPIDI_Process.cuda_aware_support_on = false; - if(rank == 0) - { - fprintf(stderr, "Error loading libcudart\n");fflush(stderr);sleep(1);exit(1); - } - } - else if(MPIDI_Process.cuda_aware_support_on) - { - if(MPIDI_Process.optimized.collectives == MPID_COLL_FCA) - if(rank == 0) - { - fprintf(stderr, "Warning: FCA is not supported with CUDA Aware support\n");fflush(stderr); - } - - MPIDI_Process.optimized.collectives = MPID_COLL_CUDA; - MPIDI_Process.optimized.select_colls = 0; - } -#endif - - /* Exit if any deprecated environment variables were specified. */ - if (found_deprecated_env_var) - { - if (rank == 0) - { - // Only rank 0 prints and exits. sleep to make sure message is sent before exiting. - // Other ranks will proceed, but will be waiting in a barrier in context create - // when the exit occurs. - fprintf (stderr, "\n"); fflush(stderr); sleep(1); - exit(1); - } - } -} - -#if TOKEN_FLOW_CONTROL -int MPIDI_set_eager_limit(unsigned int *eager_limit) -{ - char *cp; - int val; - int numTasks=MPIDI_Process.numTasks; - cp = getenv("MP_EAGER_LIMIT"); - if (cp) - { - application_set_eager_limit=1; - if ( MPIDI_atoi(cp, &val) == 0 ) - *eager_limit=val; - } - else - { - /* set default - * Number of tasks MP_EAGER_LIMIT - * ----------------- -------------- - * 1 - 256 32768 - * 257 - 512 16384 - * 513 - 1024 8192 - * 1025 - 2048 4096 - * 2049 - 4096 2048 - * 4097 & above 1024 - */ - if (numTasks < 257) *eager_limit = 32768; - else if (numTasks < 513) *eager_limit = 16384; - else if (numTasks < 1025) *eager_limit = 8192; - else if (numTasks < 2049) *eager_limit = 4096; - else if (numTasks < 4097) *eager_limit = 2048; - else *eager_limit = 1024; - - } - return 0; -} - - /******************************************************************/ - /* */ - /* Check for MP_BUFFER_MEM, if the value is not set by the user, */ - /* then set the value with the default of 64 MB. */ - /* MP_BUFFER_MEM supports the following format: */ - /* MP_BUFFER_MEM=xxM */ - /* MP_BUFFER_MEM=xxM,yyyM */ - /* MP_BUFFER_MEM=xxM,yyyG */ - /* MP_BUFFER_MEM=,yyyM */ - /* xx: pre allocated size the max. allowable value is 256 MB */ - /* the space is allocated during the initialization. */ - /* the default is 64 MB */ - /* yyy: maximum size - the maximum size to which the early arrival*/ - /* buffer can temporarily grow when the preallocated portion */ - /* of the EA buffer has been filled. */ - /* */ - /******************************************************************/ -int MPIDI_get_buf_mem(unsigned long *buf_mem,unsigned long *buf_mem_max) - { - char *cp; - int i; - int args_in_error=0; - char pre_alloc_buf[25], buf_max[25]; - char *buf_max_cp; - int pre_alloc_val=0; - unsigned long buf_max_val; - int has_error = 0; - extern int application_set_buf_mem; - - if (cp = getenv("MP_BUFFER_MEM")) { - pre_alloc_buf[24] = '\0'; - buf_max[24] = '\0'; - application_set_buf_mem=1; - if ( (buf_max_cp = strchr(cp, ',')) ) { - if ( *(++buf_max_cp) == '\0' ) { - /* Error: missing buffer_mem_max */ - has_error = 1; - } - else if ( cp[0] == ',' ) { - /* Pre_alloc value is default -- use default */ - pre_alloc_val = -1; - strncpy(buf_max, buf_max_cp, 24); - if ( MPIDI_atoll(buf_max, &buf_max_val) != 0 ) - has_error = 1; - } - else { - /* both values are present */ - for (i=0; ; i++ ) { - if ( (cp[i] != ',') && (i<24) ) - pre_alloc_buf[i] = cp[i]; - else { - pre_alloc_buf[i] = '\0'; - break; - } - } - strncpy(buf_max, buf_max_cp, 24); - if ( MPIDI_atoi(pre_alloc_buf, &pre_alloc_val) == 0 ) { - if ( MPIDI_atoll(buf_max, &buf_max_val) != 0 ) - has_error = 1; - } - else - has_error = 1; - } - } - else - { - /* Old single value format */ - if ( MPIDI_atoi(cp, &pre_alloc_val) == 0 ) - buf_max_val = (unsigned long)pre_alloc_val; - else - has_error = 1; - } - if ( has_error == 0) { - if ((int) pre_alloc_val != -1) /* MP_BUFFER_MEM=,128MB */ - *buf_mem = (int) pre_alloc_val; - if (buf_max_val > ONE_SHARED_SEGMENT) - *buf_mem = ONE_SHARED_SEGMENT; - if (buf_max_val != *buf_mem_max) - *buf_mem_max = buf_max_val; - } else { - args_in_error += 1; - TRACE_ERR("ERROR in MP_BUFFER_MEM %s(%d)\n",__FILE__,__LINE__); - return 1; - } - return 0; - } else { - /* MP_BUFFER_MEM is not specified by the user*/ - *buf_mem = BUFFER_MEM_DEFAULT; - TRACE_ERR("buffer_mem=%d buffer_mem_max=%d\n",*buf_mem,*buf_mem_max); - return 0; - } -} -#endif diff --git a/src/mpid/pamid/src/mpidi_mutex.c b/src/mpid/pamid/src/mpidi_mutex.c deleted file mode 100644 index 8411848..0000000 --- a/src/mpid/pamid/src/mpidi_mutex.c +++ /dev/null @@ -1,88 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpidi_mutex.c - * \brief ??? - */ -#include <mpidimpl.h> - - -#if MPIDI_MUTEX_L2_ATOMIC - -static inline void* -MPIDI_Mutex_initialize_l2atomics(size_t size) -{ - typedef pami_result_t (*pamix_proc_memalign_fn) (void**, size_t, size_t, const char*); - - pami_result_t rc; - pami_extension_t l2; - pamix_proc_memalign_fn PAMIX_L2_proc_memalign; - void* l2atomics = NULL; - - rc = PAMI_Extension_open(NULL, "EXT_bgq_l2atomic", &l2); - MPID_assert_always(rc == PAMI_SUCCESS); - PAMIX_L2_proc_memalign = (pamix_proc_memalign_fn)PAMI_Extension_symbol(l2, "proc_memalign"); - MPID_assert_always(PAMIX_L2_proc_memalign != NULL); - rc = PAMIX_L2_proc_memalign(&l2atomics, 64, size, NULL); - MPID_assert_always(rc == PAMI_SUCCESS); - MPID_assert_always(l2atomics != NULL); - /* printf("MPID L2 space: virt=%p HW=%p L2BaseAddress=%"PRIu64"\n", l2atomics, __l2_op_ptr(l2atomics, 0), Kernel_L2AtomicsBaseAddress()); */ - - return l2atomics; -} - -MPIDI_Mutex_t * MPIDI_Mutex_vector; -uint32_t MPIDI_Mutex_counter[MPIDI_MAX_THREADS][MPIDI_MAX_MUTEXES]; - -/** - * \brief Initialize a mutex. - * - * In this API, mutexes are acessed via indices from - * 0..MPIDI_MAX_MUTEXES. The mutexes are recursive - */ -int -MPIDI_Mutex_initialize() -{ - size_t i, j; - - MPIDI_Mutex_vector = (MPIDI_Mutex_t*)MPIDI_Mutex_initialize_l2atomics(sizeof (MPIDI_Mutex_t) * MPIDI_MAX_MUTEXES); - - for (i=0; i<MPIDI_MAX_MUTEXES; ++i) { - L2_AtomicStore(&(MPIDI_Mutex_vector[i].counter), 0); - L2_AtomicStore(&(MPIDI_Mutex_vector[i].bound), 1); - } - - for (i=0; i<MPIDI_MAX_MUTEXES; ++i) { - for (j=0; j<MPIDI_MAX_THREADS; ++j) { - MPIDI_Mutex_counter[j][i] = 0; - } - } - - return 0; -} - -#elif MPIDI_MUTEX_LLSC - -MPIDI_Mutex_t MPIDI_Mutex_vector [MPIDI_MAX_MUTEXES]; -uint32_t MPIDI_Mutex_counter[MPIDI_MAX_THREADS][MPIDI_MAX_MUTEXES]; - -#else - -pthread_mutex_t MPIDI_Mutex_lock; - -#endif diff --git a/src/mpid/pamid/src/mpidi_nbc_sched.c b/src/mpid/pamid/src/mpidi_nbc_sched.c deleted file mode 100644 index 998d03e..0000000 --- a/src/mpid/pamid/src/mpidi_nbc_sched.c +++ /dev/null @@ -1,65 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpidi_nbc_sched.c - * \brief Non-blocking collectives hooks - */ - -#include <pami.h> -#include <mpidimpl.h> - -/** - * work object for persistent advance of nbc schedules - */ -pami_work_t mpidi_nbc_work_object; - -/** - * \brief Persistent work function for nbc schedule progress - */ -pami_result_t mpidi_nbc_work_function (pami_context_t context, void *cookie) -{ - int made_progress = 0; - MPIDU_Sched_progress (&made_progress); - - return PAMI_EAGAIN; -} - -/** - * \brief Initialize support for MPIR_* nbc implementation. - * - * The MPIR_* non-blocking collectives only work if the schedule is advanced. - * This is done by posting a work function to context 0 that invokes the - * schedule progress function. - * - * Because this is a persistent work function and will negatively impact the - * performance of all other MPI operations - even when mpir non-blocking - * collectives are not used - the work function is only posted if explicitly - * requested. - */ -void MPIDI_NBC_init () -{ - if (MPIDI_Process.mpir_nbc != 0) - { - PAMI_Context_post(MPIDI_Context[0], - &mpidi_nbc_work_object, - mpidi_nbc_work_function, - NULL); - } - - return; -} diff --git a/src/mpid/pamid/src/mpidi_pami_datatype.c b/src/mpid/pamid/src/mpidi_pami_datatype.c deleted file mode 100644 index 296ef4f..0000000 --- a/src/mpid/pamid/src/mpidi_pami_datatype.c +++ /dev/null @@ -1,155 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpidi_pami_datatype.c - * \brief pami_type_t datatype hooks - */ - -#include <pami.h> -#include <mpidimpl.h> - -/** - * \brief Create PAMI datatype representation of MPI Datatype during commit. - * - * Signifcant performance improvements can be realized for one-sided communication - * utilizing the PAMI_Rput_typed and PAMI_Rget_typed interface which requires a - * PAMI representation of the MPI Datatype. - */ -void MPIDI_PAMI_datatype_commit_hook (MPI_Datatype *ptr) -{ - - /* If the PAMID optimization to utilize the PAMI_Rput_typed / PAMI_Rget_typed call for - * one-sided comm for derived types is enabled then we need to create the PAMI datatype. - */ - if (MPIDI_Process.typed_onesided == 1) { - - MPIDU_Datatype*datatype_ptr; - MPIDU_Datatype_get_ptr(*ptr, datatype_ptr); - - pami_result_t pami_dtop_result; - datatype_ptr->device_datatype = (pami_type_t *) MPL_malloc(sizeof(pami_type_t)); - pami_dtop_result = PAMI_Type_create ((pami_type_t *)datatype_ptr->device_datatype); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - - /* Flatten the non-contiguous data type into arrays describing the contiguous chunks. - */ - MPI_Aint *dt_offset_array = (MPI_Aint *) MPL_malloc(datatype_ptr->max_contig_blocks * sizeof(MPI_Aint)); - MPI_Aint *dt_size_array = (MPI_Aint *) MPL_malloc(datatype_ptr->max_contig_blocks * sizeof(MPI_Aint)); - MPI_Aint dt_array_len = datatype_ptr->max_contig_blocks; - int rc = MPIR_Type_flatten(*ptr, dt_offset_array, dt_size_array, &dt_array_len); - - /* Build the PAMI datatype adding one contiguous chunk at a time with the PAMI_Type_add_simple - * interface. - */ - int i; - - for (i=0;i<dt_array_len;i++) { - size_t num_bytes_this_entry = dt_size_array[i]; - size_t cursor_offset; - if (i == 0) - cursor_offset = (size_t) dt_offset_array[i]; - else - cursor_offset = (size_t) dt_offset_array[i] - (size_t)dt_offset_array[i-1]; - pami_dtop_result = PAMI_Type_add_simple (*(pami_type_t*)(datatype_ptr->device_datatype), num_bytes_this_entry, cursor_offset, 1, 0); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - } - - /* Complete the PAMI datatype and free arrays. - */ - pami_dtop_result = PAMI_Type_complete (*(pami_type_t*)(datatype_ptr->device_datatype),1); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - MPL_free(dt_offset_array); - MPL_free(dt_size_array); - } - return; -} - -/** - * \brief Destroy PAMI datatype representation of MPI Datatype. - * - */ -void MPIDI_PAMI_datatype_destroy_hook (MPIDU_Datatype*ptr) -{ - /* If a PAMI datatype was created, destroy it if this is the - * last reference to the MPIDU_Datatypeptr. - */ - if ((MPIDI_Process.typed_onesided == 1) && (ptr->is_committed)) { - if (ptr->device_datatype) { - pami_result_t pami_dtop_result; - pami_dtop_result = PAMI_Type_destroy ((pami_type_t *)ptr->device_datatype); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - MPL_free(ptr->device_datatype); - } - } -} - -/** - * \brief Create PAMI datatype representation of MPI Datatype during dup. - * - * Signifcant performance improvements can be realized for one-sided communication - * utilizing the PAMI_Rput_typed and PAMI_Rget_typed interface which requires a - * PAMI representation of the MPI Datatype. - */ -void MPIDI_PAMI_datatype_dup_hook (MPI_Datatype *ptr) -{ - - /* If the PAMID optimization to utilize the PAMI_Rput_typed / PAMI_Rget_typed call for - * one-sided comm for derived types is enabled then we need to create the PAMI datatype. - */ - if (MPIDI_Process.typed_onesided == 1) { - - MPIDU_Datatype*datatype_ptr; - MPIDU_Datatype_get_ptr(*ptr, datatype_ptr); - - pami_result_t pami_dtop_result; - datatype_ptr->device_datatype = (pami_type_t *) MPL_malloc(sizeof(pami_type_t)); - pami_dtop_result = PAMI_Type_create ((pami_type_t *)datatype_ptr->device_datatype); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - - /* Flatten the non-contiguous data type into arrays describing the contiguous chunks. - */ - MPI_Aint *dt_offset_array = (MPI_Aint *) MPL_malloc(datatype_ptr->max_contig_blocks * sizeof(MPI_Aint)); - MPI_Aint *dt_size_array = (MPI_Aint *) MPL_malloc(datatype_ptr->max_contig_blocks * sizeof(MPI_Aint)); - MPI_Aint dt_array_len = datatype_ptr->max_contig_blocks; - int rc = MPIR_Type_flatten(*ptr, dt_offset_array, dt_size_array, &dt_array_len); - - /* Build the PAMI datatype adding one contiguous chunk at a time with the PAMI_Type_add_simple - * interface. - */ - int i; - - for (i=0;i<dt_array_len;i++) { - size_t num_bytes_this_entry = dt_size_array[i]; - size_t cursor_offset; - if (i == 0) - cursor_offset = (size_t) dt_offset_array[i]; - else - cursor_offset = (size_t) dt_offset_array[i] - (size_t)dt_offset_array[i-1]; - pami_dtop_result = PAMI_Type_add_simple (*(pami_type_t*)(datatype_ptr->device_datatype), num_bytes_this_entry, cursor_offset, 1, 0); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - } - - /* Complete the PAMI datatype and free arrays. - */ - pami_dtop_result = PAMI_Type_complete (*(pami_type_t*)(datatype_ptr->device_datatype),1); - MPIR_Assert(pami_dtop_result == PAMI_SUCCESS); - MPL_free(dt_offset_array); - MPL_free(dt_size_array); - } - return; -} diff --git a/src/mpid/pamid/src/mpidi_util.c b/src/mpid/pamid/src/mpidi_util.c deleted file mode 100644 index ec88346..0000000 --- a/src/mpid/pamid/src/mpidi_util.c +++ /dev/null @@ -1,2057 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpid_util.c - * \brief Normal job startup code - */ - - -/* - * \brief - */ -#include <dlfcn.h> -#include <sys/time.h> -#include <sys/types.h> -#include <sys/stat.h> -#include <errno.h> -#include <limits.h> -#include <nl_types.h> -#include <mpidimpl.h> -#include "mpidi_util.h" - -#define PAMI_TUNE_MAX_ITER 2000 -/* Short hand for sizes */ -#define ONE (1) -#define ONEK (1<<10) -#define ONEM (1<<20) -#define ONEG (1<<30) - -#define PAMI_ASYNC_EXT_ATTR 2000 - -#if CUDA_AWARE_SUPPORT -void * pamidCudaPtr = NULL; -#endif -#if (MPIDI_PRINTENV || MPIDI_STATISTICS || MPIDI_BANNER) -MPIDI_printenv_t *mpich_env=NULL; -extern char* mp_euilib; -char mp_euidevice[20]; -extern pami_extension_t pe_extension; -extern pamix_extension_info_t PAMIX_Extensions; - typedef enum { - /* Attribute usage : type : default : description */ - PAMI_ASYNC_ATTR - = PAMI_ASYNC_EXT_ATTR, - PAMI_CONTEXT_TIMER_INTERVAL, /**< U: size_t : N/A : current timer interval in PAMI context */ - } pamix_attribute_name_async_t; - - -void MPIDI_Set_mpich_env(int rank, int size) { - static char polling_buf[32]=""; - int rc; - pami_configuration_t config; - - mpich_env->this_task = rank; - mpich_env->nprocs = size; - mpich_env->eager_limit=MPIDI_Process.pt2pt.limits.application.eager.remote; - mpich_env->use_token_flow_control=MPIDI_Process.is_token_flow_control_on; - mpich_env->mp_statistics=MPIDI_Process.mp_statistics; - if (mpich_env->polling_interval == 0) { - memset(&config, 0, sizeof(config)); - config.name = (pami_attribute_name_t)PAMI_CONTEXT_TIMER_INTERVAL; - rc= PAMI_Context_query(MPIDI_Context[0], &config, 1); - mpich_env->polling_interval = config.value.intval;; - sprintf(polling_buf, "MP_POLLING_INTERVAL=%d", - mpich_env->polling_interval); /* microseconds */ - rc = putenv(polling_buf); - } - if (mpich_env->retransmit_interval == 0) { - memset(&config, 0, sizeof(config)); - config.name = (pami_attribute_name_ext_t)PAMI_CONTEXT_RETRANSMIT_INTERVAL; - rc= PAMI_Context_query(MPIDI_Context[0], &config, 1); - mpich_env->retransmit_interval = config.value.intval; - sprintf(polling_buf, "MP_RETRANSMIT_INTERVAL=%d", - mpich_env->retransmit_interval); /* microseconds */ - rc = putenv(polling_buf); - } - mpich_env->buffer_mem=MPIDI_Process.mp_buf_mem; - mpich_env->buffer_mem_max=MPIDI_Process.mp_buf_mem_max; -} - - - -void MPIDI_Setup_networkenv() -{ - char *network; - char delimiter; - int rc; - pami_configuration_t config; - - - delimiter=':'; - mpich_env->instances = 1; - mpich_env->strip_type = NO_STRIPING; - mpich_env->transport_type = IS_IP; - mpich_env->network_string = NULL; - -/* - * Always try the MPI environment variables first - * then the PAMI env variable - * then query the PAMI network environment string - */ - if ((network = getenv("MP_CHILD_INET_ADDR")) != NULL ) { - mpich_env->transport_type = IS_IP; - } else if ((network = getenv("MP_MPI_NETWORK")) != NULL) { - mpich_env->transport_type = IS_US; - } else { - network=NULL; - memset(&config,0, sizeof(config)); - config.name = (pami_attribute_name_t)PAMI_CONTEXT_NETWORK; - rc= PAMI_Context_query(MPIDI_Context[0], &config, 1); - network=(char *) config.value.chararray; - if ((rc != PAMI_SUCCESS) || (network== NULL)) { - TRACE_ERR("PAMI network query returns %d", rc); - network=NULL; - return; - } else { - if ( strchr(network, ':') || - strchr(network, '.')) { - mpich_env->transport_type = IS_IP; - } else { - mpich_env->transport_type = IS_US; - } - } - } - - if ( NULL != network ) { - mpich_env->network_string = (char *) MPL_malloc(strlen(network)+1); - if( NULL != mpich_env->network_string ) { - memcpy(mpich_env->network_string,network,strlen(network)+1); - } - mpich_env->instances = atoi(network+1); - if (mpich_env->instances > 1 ) { - mpich_env->strip_type = IS_PACKET_STRIPING; - } - } - /* - * These should match the PAMI defaults. They are provided so that the - * mpi_printenv variable can print them out. - */ - if ( mpich_env->retransmit_interval == 0 ) { - memset(&config, 0, sizeof(config)); - config.name = (pami_attribute_name_t)PAMI_CONTEXT_RETRANSMIT_INTERVAL; - rc= PAMI_Context_query(MPIDI_Context[0], &config, 1); - mpich_env->retransmit_interval = config.value.intval; - } - -} - - -int MPIDI_Update_mpenv() -{ - char *buf; - int rc=0; - int i, val; /* for loop index */ - long bdfrsize; - int SendBufSize; - int CopySendBufSize; - char *cp,*p; - int infolevel, max_pkt_size; - uint tmp_val; - size_t bulkXfer=0; - pami_configuration_t *pami_config,config; - - MPIDI_Setup_networkenv(); - memset(&config,0, sizeof(config)); - config.name = PAMI_DISPATCH_SEND_IMMEDIATE_MAX; - rc = PAMI_Dispatch_query(MPIDI_Context[0], (size_t)0, &config, 1); - if ( rc == PAMI_SUCCESS ) { - TRACE_ERR("PAMI_DISPATCH_SEND_IMMEDIATE_MAX=%d.\n", config.value.intval, rc); - SendBufSize = config.value.intval; - } else { - TRACE_ERR((" Attention: PAMI_Client_query(DISPATCH_SEND_IMMEDIATE_MAX=%d) rc=%d\n", - config.name, rc)); - SendBufSize = 256; /* This is PAMI SendBufSize. */ - } - CopySendBufSize = SendBufSize - sizeof(MPIDI_MsgInfo); - mpich_env->copySendBufSize = CopySendBufSize; - - pami_config = (pami_configuration_t *) MPL_malloc(10 * sizeof(pami_configuration_t)); - memset((void *) pami_config,0, sizeof(pami_config)); - pami_config[0].name = (pami_attribute_name_t)PAMI_CONTEXT_MAX_PKT_SZ; - pami_config[1].name = (pami_attribute_name_t)PAMI_CONTEXT_RFIFO_SZ; - pami_config[2].name = (pami_attribute_name_t)PAMI_CONTEXT_SHM_ENABLED; - pami_config[3].name = (pami_attribute_name_t)PAMI_CONTEXT_BULK_XFER; - pami_config[4].name = (pami_attribute_name_t)PAMI_CONTEXT_BULK_MIN_MSG_SZ; - rc= PAMI_Context_query(MPIDI_Context[0], pami_config, 5); - if (rc != PAMI_SUCCESS) { - printf("ERROR PAMI_Context_query() failed rc=%d\n",rc); fflush(stdout); - return rc; - } - max_pkt_size = pami_config[0].value.intval; - mpich_env->recv_fifo_sz = pami_config[1].value.intval; - mpich_env->max_pkt_size = max_pkt_size; - mpich_env->max_pkts_out = 31; - mpich_env->use_shmem = pami_config[2].value.intval; - bulkXfer=pami_config[3].value.intval; - if (bulkXfer) { - mpich_env->use_rdma=PAMI_HINT_ENABLE; - } else { - /* No bulk xfer in PAMI */ - if (mpich_env->this_task == 0) { - TRACE_ERR("MPI-PAMI: Attention: No bulk xfer for the job\n"); - } - mpich_env->use_rdma=PAMI_HINT_DISABLE; /* in sync with PAMI */ - } - /* obtain minimum message size for bulk xfer (rdma) */ - mpich_env->rdma_min_msg_size = (long) pami_config[4].value.intval; - MPL_free(pami_config); -} - - -/* Print Environment */ -/* _mpidi_printenv collects the relevant MPI and internal variables to task 0 */ -/* and then prints out the differences */ -/* Returns: 0 - OK; possibly some ATTENTION messages issued; */ -/* 1 - ERROR; user must call exit(1), or risk the application hanging */ - - - - -#define MATCHI(x,y) { \ - first = 0; last = task_count-1; cflag = ' '; \ - for (i=1;i<=task_count;i++) { \ - if ((i==task_count) || (gatherer[i].x != gatherer[first].x)) { \ - last = i-1; \ - if (last != task_count-1) cflag = '*'; \ - printf("Task %1c%4d-%4d:%s %d\n", cflag,first, last, y, \ - gatherer[first].x); \ - first = i; \ - } \ - } \ - } -#define MATCHLL(x,y) { \ - first = 0; last = task_count-1; cflag = ' '; \ - for (i=1;i<=task_count;i++) { \ - if ((i==task_count) || (gatherer[i].x != gatherer[first].x)) { \ - last = i-1; \ - if (last != task_count-1) cflag = '*'; \ - printf("Task %1c%4d-%4d:%s %lld\n", cflag,first, last, \ - y, gatherer[first].x); \ - first = i; \ - } \ - } \ - } -#define MATCHD(x,y) { \ - first = 0; last = task_count-1; cflag = ' '; \ - for (i=1;i<=task_count;i++) { \ - if ((i==task_count) || (gatherer[i].x != gatherer[first].x)) { \ - last = i-1; \ - if (last != task_count-1) cflag = '*'; \ - printf("Task %1c%4d-%4d:%s %lf\n", cflag, first, last, \ - y, gatherer[first].x); \ - first = i; \ - } \ - } \ - } - -#define MATCHC(x,y,m) { \ - first = 0; last = task_count-1; cflag = ' '; \ - gatherer[first].x[m-1]='\0'; \ - for (i=1;i<=task_count;i++) { \ - if (i<task_count) gatherer[i].x[m-1]='\0'; \ - if ((i==task_count) || strncmp(gatherer[i].x, gatherer[first].x, m)) {\ - last = i-1; \ - if (last != task_count-1) cflag = '*'; \ - printf("Task %1c%4d-%4d:%s %s\n", cflag, first, last, y, \ - gatherer[first].x); \ - first = i; \ - } \ - } \ - } -#define MATCHB(x,y) { \ - first = 0; last = task_count-1; cflag = ' '; \ - for (i=1;i<=task_count;i++) { \ - if ((i==task_count) || (gatherer[i].x != gatherer[first].x)) { \ - last = i-1; \ - if (last != task_count-1) cflag = '*'; \ - if (gatherer[first].x > 0) { \ - printf("Task %1c%4d-%4d:%s %s\n", cflag, first, last, \ - y, "YES"); \ - } else if (gatherer[first].x == 0) { \ - printf("Task %1c%4d-%4d:%s %s\n", cflag, first, \ - last, y, "NO"); \ - } else { \ - printf("Task %1c%4d-%4d:%s %d\n", cflag, first, last, \ - y, gatherer[first].x); \ - } \ - first = i; \ - } \ - } \ - } -/* A modification on MATCHB, it is used to check the environment variables - * that take one of YES, NO or CONFIRM, This is used in particular for checking the value - * passed to MP_SINGLE_THREAD */ -#define MATCHT(x,y) { \ - first = 0; last = task_count-1; cflag = ' '; \ - for (i=1;i<=task_count;i++) { \ - if ((i==task_count) || (gatherer[i].x != gatherer[first].x)) { \ - last = i-1; \ - if (last != task_count-1) cflag = '*'; \ - if (gatherer[first].x == 0) { \ - printf("Task %1c%4d-%4d:%s %s\n", cflag, first, last, y, \ - "NO"); \ - } else if (gatherer[first].x == 1) { \ - printf("Task %1c%4d-%4d:%s %s\n", cflag, first, last, \ - y, "YES"); \ - } else { \ - printf("Task %1c%4d-%4d:%s %d\n", cflag, first, last, y, \ - gatherer[first].x); \ - } \ - first = i; \ - } \ - } \ - } - -#define NOTAPPLICABLE(y) { \ - printf("Task %4d-%4d:%s NA\n", 0, task_count-1, y); \ - } - - - -/* - * Parse Network string value - * Format = @instance_number:netAddr,adapter_type:..... - * or - * Format = @instance_number;netAddr,adapter_type;..... - * netAddr is for either Inet address (IP) or window ID (US) - * adapter is for adapter type - * - * Newwork_ID is exported by POE - * - * This function will fill in the given buffers as much as possible - * will start to drop the (whole) info when the buffers are getting - * full. - */ -void MPIDI_Get_net_param(char *cp, int MPI_only, int my_taskid, int total_task, - char *netAddr, long addrSize, - char *adapter, long adptSize, - char *networkId, long networkIdSize) -{ - /* parse Network string, works with either user space or IP */ - /* removing the window numbers but keeping the device names */ - char *cp1, *cp2, *cp3, *dummy1, *dummy2, *dummy3; - long nc, addrSpace=addrSize, addrLen=0, adptSpace=adptSize, adptLen=0; - int taskid, inst, i, n_comma; - char delimiter = ':'; - char msgBuf[80]; - - *netAddr = '\0'; /* Make sure strcat starts */ - *adapter = '\0'; /* from the beginning. */ - - if ( strchr(cp, ';') ) { - delimiter = ';'; - } - - - if ( *cp == '@' ) { - /* PE 4.1: MP_MPI_NETWORK = @inst:addr,adapter:...,... */ - /* Skip the instance number and get to the first colon */ - cp1 = strchr(cp,delimiter); - - while (cp1) { - cp1++; /* Next address */ - cp2 = strchr(cp1,',') + 1; /* Address ends at a comma */ - nc = cp2 - cp1; /* Add 1 to pick up the comma */ - if ( nc <= addrSpace ) { /* Yes test nc <= addrSpace */ - strncat(netAddr, cp1, nc); /* We will take care of the */ - addrLen += nc; /* last 'comma' later. */ - addrSpace -= nc; - } - else - break; - if ( (cp1 = strchr(cp2,delimiter)) != NULL ) { /* Adapter ends at a colon */ - nc = cp1 - cp2 + 1; /* Add 1 to pick up colon */ - if ( nc < adptSpace ) { - strncat(adapter, cp2, nc); - adptLen += nc; - adptSpace -= nc; - } - else { - strncat(adapter, cp2, adptSpace-1); - adptLen += (adptSpace-1); - break; - } - } - else { - /* Last one */ - nc = strlen(cp2); - if ( nc < adptSpace ) { - strcat(adapter, cp2); - adptLen += nc; - } - else { - strncat(adapter, cp2, adptSpace-1); - adptLen += (adptSpace-1); - } - } - } - /* Terminate netAddr and adapter with NULL */ - if ( addrLen > 1 ) netAddr[addrLen-1] = '\0'; /* get rid of the last comma */ - if ( adptLen > 0 ) { - if ( adapter[adptLen-1] == delimiter ) adapter[adptLen-1] = '\0'; - else adapter[adptLen] = '\0'; - } - } else { - /* MP_MPI_NETWORK = window_number:device_number */ - cp1 = strchr(cp,':'); - nc = cp1 - cp; /* Don't include ':' */ - strncpy(netAddr, cp, nc); - netAddr[nc] = '\0'; - strcpy(adapter, cp1+1); - } -} - - -int MPIDI_Print_mpenv(int rank,int size) -{ - MPI_Comm comm = MPI_COMM_WORLD; - MPIR_Comm *comm_ptr; - MPIDI_printenv_t sender; - MPIDI_printenv_t *gatherer = NULL; - int mytask; - int task_count; - MPIDI_TransportType transport_type; - MPIDI_StripingType striping_type; - int instances; - int is_mpi=1; - char *cp; - char *cp1; - int i, j, k; - int first, last; - int len, tmp_val; - char *userbuf = NULL; - int *recvcount = NULL, *recvdispl = NULL; - char **recvwork = NULL; - int max_output_size = 0; - FILE *fd; - char *ts; - char *ck; - int ssi = 0; - int current_size; - int increment; - int m, m1; - char cflag; - int rc; - pami_configuration_t *pami_config, config; - char *p; - char popenstr[]={ "/bin/ksh -c " }; - char *popenptr; - char tempstr[128]; - int mpi_errno; - int errflag=0; - - MPIDI_Set_mpich_env(rank,size); - memset(&sender,0,sizeof(MPIDI_printenv_t)); - MPIDI_Update_mpenv(); - /* decoding strings */ - char *clock_string[] = { -#ifndef __LINUX__ - "AIX", -#else - "OS", -#endif - "Switch" }; - char *develop_string[] = { - "Minimum", - "Normal", - "Debug", - "Develop" }; - char *transport_type_string[] = { - "ip", - "us" }; - char *striping_type_set[] = { - "Striping off", - "Packet striping on" }; - - /* Get MPI Library Version */ - MPIDI_Banner((char*)&sender.version); - /* Remove the trailing newline character */ - for (i=strlen(sender.version);i>0;i--) { - if (sender.version[i-1] == '\n') sender.version[i-1] = '\0'; } - - gethostname(sender.hostname, MAXHOSTNAMELEN+1); - /* Get task geometry information */ - cp = getenv("MP_PARTITION"); /* MP_PARTITION is always there */ - sender.partition = atol(cp); /* Checking is not needed. */ - sender.pid = getpid(); - mytask = mpich_env->this_task; - sender.nprocs = mpich_env->nprocs; - sender.single_thread = mpich_env->single_thread; - sender.max_pkts_out = mpich_env->max_pkts_out; - sender.recv_fifo_sz = mpich_env->recv_fifo_sz; - task_count = sender.nprocs; - strcpy(sender.wait_mode,"NOT SET"); /* will be updated in the future */ - strcpy(sender.clock_source,"OS"); - cp = getenv("MP_NODES"); - if (cp) strncpy(sender.nnodes, cp, 8); - else strcpy(sender.nnodes, "NOT SET"); - cp = getenv("MP_TASKS_PER_NODE"); - if (cp) strncpy(sender.tasks_node, cp, 8); - else strcpy(sender.tasks_node, "NOT SET"); - - if(cp=getenv("MP_EUIDEVELOP")) { - strcpy((void *) &sender.develop,(void *)cp); - } else - memcpy(&sender.develop,"NOT SET ",8); - - if(cp=getenv("MP_EUIDEVICE")) { - strcpy((void *) &mp_euidevice,(void *)cp); - } else - memcpy(&mp_euidevice,"NOT SET ",8); - strncpy(sender.device, mp_euidevice, 8); - - /* Get protocol info */ - transport_type = mpich_env->transport_type; - switch (transport_type) - { - case IS_IP: - strcpy(sender.euilib, transport_type_string[transport_type]); - cp = NULL; - cp = (char *) mpich_env->network_string; - if (cp) { - strcpy(sender.window_id, "NA"); - MPIDI_Get_net_param(cp, is_mpi, mytask, task_count, - tempstr, sizeof(tempstr), - sender.adapter_type, sizeof(sender.adapter_type), - sender.network_id, sizeof(sender.network_id)); - /* Override network_id when network device is en0 or ml0 */ - if ( !strcasecmp(sender.device, "en0") || - !strcasecmp(sender.device, "ml0") ) { - strcpy(sender.network_id, "NA"); - } - } else { - strcpy(sender.window_id, "NOT_SET"); - strcpy(sender.adapter_type, "NOT SET"); - strcpy(sender.network_id, "NOT SET"); - } - break; - case IS_US: - strcpy(sender.euilib, transport_type_string[transport_type]); - cp = NULL; - cp = (char *) mpich_env->network_string; - if (cp) { - MPIDI_Get_net_param(cp, is_mpi, mytask, task_count, - sender.window_id, sizeof(sender.window_id), - sender.adapter_type, sizeof(sender.adapter_type), - sender.network_id, sizeof(sender.network_id)); - } else { - strcpy(sender.window_id, "NOT SET"); - strcpy(sender.adapter_type, "NOT SET"); - strcpy(sender.network_id, "NOT SET"); - } - break; - default: - sprintf(tempstr, "_mpi_printenv: Get unknown transport type [%d]!\n", transport_type); - TRACE_ERR("%s \n",tempstr); - strcpy(sender.euilib, "UNKNOWN"); - strcpy(sender.window_id, "UNKNOWN"); - strcpy(sender.adapter_type, "UNKNOWN"); - strcpy(sender.network_id, "UNKNOWN"); - break; - } - /* Get run time setup */ - cp = getenv("MP_MSG_API"); - if (cp) strncpy(sender.protocol, cp, 16); - else strcpy(sender.protocol, "MPI"); - cp = getenv("LIBPATH"); - if (cp) strncpy(sender.libpath, cp, MAXPATHLEN-1); - else strcpy(sender.libpath, "NOT SET"); - getcwd(sender.directory, FILENAME_MAX+1); - - if(sizeof(void*) == 4) - sender.mode_64 = 0; - else - sender.mode_64 = 1; - - sender.threaded = 1; /* Always 1 */ - cp = getenv("AIXTHREAD_SCOPE"); - if (cp) strncpy(sender.thread_scope, cp, 8); - else strcpy(sender.thread_scope,"NOT SET"); - - cp = getenv("MP_CPU_USE"); - if (cp) strncpy(sender.cpu_use, cp, 10); - else strcpy(sender.cpu_use, "NOT SET"); - cp = getenv("MP_ADAPTER_USE"); - if (cp) strncpy(sender.adapter_use, cp, 10); - else strcpy(sender.adapter_use, "NOT SET"); -#ifndef LINUX - cp = getenv("MP_PRIORITY"); - if (cp) strncpy(sender.priority, cp, 24); - else strcpy(sender.priority, "NOT SET"); -#else - strcpy(sender.priority, "NA"); -#endif - cp = getenv("MP_TIMEOUT"); - if (cp) { - sender.timeout = atoi(cp); - if(sender.timeout <= 0) { - sender.timeout = 150; /* default MP_TIMEOUT is 150s */ - } - } else { - sender.timeout = 150; - } - - sender.interrupts = mpich_env->interrupts; - sender.mp_statistics = mpich_env->mp_statistics; - sender.polling_interval = mpich_env->polling_interval; - sender.eager_limit = mpich_env->eager_limit; - sender.use_token_flow_control=MPIDI_Process.is_token_flow_control_on; - sender.retransmit_interval = mpich_env->retransmit_interval; - sender.buffer_mem = mpich_env->buffer_mem; - sender.buffer_mem_max = mpich_env->buffer_mem_max; - - /* Get shared memory */ - sender.shmem_pt2pt = MPIDI_Process.shmem_pt2pt; - sender.use_shmem = mpich_env->use_shmem; -#ifndef LINUX - cp = getenv("MEMORY_AFFINITY"); - if ( cp ) strncpy(sender.mem_affinity, cp, 8); - else strcpy(sender.mem_affinity, "NOT SET"); -#else - strcpy(sender.mem_affinity, "NA"); -#endif - sender.instances = mpich_env->instances; - strcpy(sender.striping_type, striping_type_set[(mpich_env->strip_type)]); - - - pami_config = (pami_configuration_t *) MPL_malloc(10 * sizeof(pami_configuration_t)); - memset((void *) pami_config,0, sizeof(pami_config)); - pami_config[0].name = (pami_attribute_name_t)PAMI_CONTEXT_ACK_THRESH; - pami_config[1].name = (pami_attribute_name_t)PAMI_CONTEXT_REXMIT_BUF_CNT; - pami_config[2].name = (pami_attribute_name_t)PAMI_CONTEXT_REXMIT_BUF_SZ; - pami_config[3].name = (pami_attribute_name_t)PAMI_CONTEXT_RC_MAX_QP; - pami_config[4].name = (pami_attribute_name_t)PAMI_CONTEXT_RC_USE_LMC; - rc= PAMI_Context_query(MPIDI_Context[0], pami_config, 5); - if (rc != PAMI_SUCCESS) { - printf("mp_printenv ERROR PAMI_Context_query() failed rc=%d\n",rc); fflush(stdout); - return rc; - } - sender.ack_thresh = pami_config[0].value.intval; - sender.rexmit_buf_cnt = pami_config[1].value.intval; - sender.rexmit_buf_size = pami_config[2].value.intval; - sender.rc_max_qp = pami_config[3].value.intval; - if (pami_config[4].value.intval) - strcpy(sender.rc_qp_use_lmc, "YES"); - else - strcpy(sender.rc_qp_use_lmc, "NO"); - MPL_free(pami_config); - - /* - Begin Bulk transfer/RDMA & IB settings */ - if ( mpich_env->use_rdma == PAMI_HINT_DISABLE ) - strcpy(sender.use_bulk_xfer, "NO"); - else - strcpy(sender.use_bulk_xfer, "YES"); - - sprintf(sender.bulk_min_msg_size, "%d", mpich_env->bulk_min_msg_size); - - - /* M28,M32 -- End Bulk transfer/RDMA & IB settings */ - - if ( cp = getenv("MP_DEBUG_NOTIMEOUT") ) - strcpy(sender.debug_notimeout, cp); - else - strcpy(sender.debug_notimeout, "NOT_SET"); - - sender.statistics = mpich_env->statistics; - cp = getenv("MP_STDINMODE"); - if ( cp ) strncpy(sender.stdinmode, cp, 10); - else strcpy(sender.stdinmode, "NOT SET"); - cp = getenv("MP_STDOUTMODE"); - if ( cp ) strncpy(sender.stdoutmode, cp, 10); - else strcpy(sender.stdoutmode, "NOT SET"); - - - /* Check if there are any environment variables of the form MP_S_, and count them */ - { - extern char **environ; - char **envp = environ; - int env_count = 0; - while (*envp) { - if (!strncmp(*envp, "MP_S_",5)) env_count++; - if (!strncmp(*envp, "MP_CCL_",7)) env_count++; /* MP_CCL_TIMING is also service */ - envp++; - } - sender.service_variables = env_count; - } - - if (mytask == 0) { /* allocate a receive buffer for the gather of the base structure */ - gatherer = (MPIDI_printenv_t*) MPL_malloc(task_count * sizeof(MPIDI_printenv_t)); - if (!gatherer) { - TRACE_ERR("_mpi_printenv gatherer MPL_malloc failed rc=%d\n",rc); - return 1; - } - memset(gatherer,0,task_count*sizeof(MPIDI_printenv_t)); - } - - - mpi_errno = MPI_SUCCESS; - MPIR_Comm_get_ptr( comm, comm_ptr ); - mpi_errno = MPIR_Gather_impl(&sender, sizeof(MPIDI_printenv_t), MPI_BYTE, gatherer, - sizeof(MPIDI_printenv_t),MPI_BYTE, 0,comm_ptr, - (int *) &errflag); - if (mpi_errno) { - /* for communication errors, just record the error but continue */ - errflag = TRUE; - } - - /* work through results and compare */ - - if (mytask == 0) { - MATCHC(version,"Library:",128); - MATCHC(hostname,"Hostname:",MAXHOSTNAMELEN+1); - MATCHI(partition,"Job ID (MP_PARTITION):"); - MATCHI(nprocs,"Number of Tasks (MP_PROCS):"); - MATCHC(nnodes,"Number of Nodes (MP_NODES):",8); - MATCHC(tasks_node,"Number of Tasks per Node (MP_TASKS_PER_NODE):",8); - MATCHC(euilib,"Library Specifier (MP_EUILIB):",8); - MATCHC(adapter_type,"Adapter Name:",32); - MATCHC(window_id,"Window ID:",128); - if ( getenv("MP_S_STRIPING_DBG") ) MATCHC(network_id,"Network ID:",32); - MATCHC(device,"Device Name (MP_EUIDEVICE):",8); -#ifndef LINUX /* M27 */ - MATCHI(instances,"Window Instances (MP_INSTANCES * # of networks):"); - MATCHC(striping_type,"Striping Setup:",40); -#endif - MATCHC(protocol,"Protocols in Use (MP_MSG_API):",16); - MATCHC(libpath,"Effective Libpath (LIBPATH):",MAXPATHLEN); - MATCHC(directory,"Current Directory:",FILENAME_MAX+1); - MATCHB(mode_64,"64 Bit Mode:"); -#ifndef LINUX /* M27 */ - MATCHB(threaded,"Threaded Library:"); - MATCHC(thread_scope,"Requested Thread Scope (AIXTHREAD_SCOPE):",8); -#endif - MATCHC(cpu_use,"CPU Use (MP_CPU_USE):",10); - MATCHC(adapter_use,"Adapter Use (MP_ADAPTER_USE):",10); - MATCHC(clock_source,"Clock Source (MP_CLOCK_SOURCE):",8); - MATCHC(priority,"Priority Class (MP_PRIORITY):",24); - MATCHI(timeout,"Connection Timeout (MP_TIMEOUT/sec):"); - MATCHB(interrupts,"Adapter Interrupts Enabled (MP_CSS_INTERRUPT):"); - MATCHI(polling_interval,"Polling Interval (MP_POLLING_INTERVAL/usec):"); - MATCHI(buffer_mem,"Buffer Memory (MP_BUFFER_MEM/Bytes):"); - MATCHLL(buffer_mem_max,"Max. Buffer Memory (MP_BUFFER_MEM_MAX/Bytes):"); - MATCHI(eager_limit,"Message Eager Limit (MP_EAGER_LIMIT/Bytes):"); - MATCHI(use_token_flow_control,"Use token flow control:"); - MATCHC(wait_mode,"Message Wait Mode(MP_WAIT_MODE):",8); - MATCHI(retransmit_interval,"Retransmit Interval (MP_RETRANSMIT_INTERVAL/count):"); - MATCHB(use_shmem,"Shared Memory Enabled (MP_SHARED_MEMORY):"); - MATCHB(shmem_pt2pt,"Intranode pt2pt Shared Memory Enabled (MP_SHMEM_PT2PT):"); - MATCHB(mp_statistics,"Statistics Collection Enabled (MP_STATISTICS):"); - MATCHC(mem_affinity,"MEMORY_AFFINITY:",8); - MATCHT(single_thread,"Single Thread Usage(MP_SINGLE_THREAD):"); - MATCHI(recv_fifo_sz,"DMA Receive FIFO Size (Bytes):"); - MATCHI(max_pkts_out,"Max outstanding packets:"); - MATCHC(develop,"Develop Mode (MP_EUIDEVELOP):",16); - MATCHC(stdinmode,"Standard Input Mode (MP_STDINMODE):",12); - MATCHC(stdoutmode,"Standard Output Mode (MP_STDOUTMODE):",12); - MATCHI(service_variables,"Number of Service Variables set (MP_S_*):"); - printf("--------------------End of MPI Environment Report-------------------------\n"); - fflush(stdout); - } /* task 0 specific stuff */ - - - if (mytask == 0) { - MPL_free(gatherer); - } - - return 0; -} - - /* ------------------------- */ - /* if (MP_STATISTICS==yes) */ - /* print statistics data */ - /* ------------------------- */ -void MPIDI_print_statistics() { - if ((MPIDI_Process.mp_statistics) || - (MPIDI_Process.mp_printenv)) { - if (MPIDI_Process.mp_statistics) { - MPIDI_Statistics_write(stdout); - if (mpid_statp) { - MPL_free(mpid_statp); - mpid_statp=NULL; - } - } - if (MPIDI_Process.mp_printenv) { - if (mpich_env) { - MPL_free(mpich_env); - mpich_env=NULL; - } - } - } -} - -#endif /* MPIDI_PRINTENV || MPIDI_STATISTICS */ - -/** - * \brief validate whether a lpid is in a given group - * - * Searches the group lpid list for a match. - * - * \param[in] lpid World rank of the node in question - * \param[in] grp Group to validate against - * \return TRUE is lpid is in group - */ - -int MPIDI_valid_group_rank(int lpid, MPIR_Group *grp) { - int size = grp->size; - int z; - - for (z = 0; z < size && - lpid != grp->lrank_to_lpid[z].lpid; ++z); - return (z < size); -} - -/****************************************************************/ -/* function MPIDI_uppers converts a passed string to upper case */ -/****************************************************************/ -void MPIDI_toupper(char *s) -{ - int i; - if (s != NULL) { - for(i=0;i<strlen(s);i++) s[i] = toupper(s[i]); - } -} - -/* - ----------------------------------------------------------------- - Name: MPID_scan_str() - - Function: Scan a flag string for 2 out of 3 possible - characters (K, M, G). Return a 1 if neither - character is found otherwise return the character - along with a buffer containing the string without - the character. - value are valid. If they are valid, the - multiplication of the number and the units - will be returned as an unsigned int. If the - number and units are invalid, a 1 will be returned. - - Description: search string for character or end of string - if string contains either entered character - check which char it is, set multiplier - if no chars found, return error - - Parameters: A0 = MPIDI_scan_str(A1, A2, A3, A4, A5) - - A1 string to scan char * - A2 first char to scan for char - A3 second char to scan for char - A4 multiplier char * - A5 returned string char * - - A0 Return Code int - - - Return Codes: 0 OK - 1 input chars not found - ------------------------------------------------------------ -*/ -int MPIDI_scan_str(char *my_str, char fir_c, char sec_c, char *multiplier, char *tempbuf) -{ - int str_ptr; /*index counter into string*/ - int found; /*indicates whether one of input chars found*/ - int len_my_str; /*length of string with size and units*/ - - str_ptr = 0; /*start at beginning of string*/ - found = 0; /*no chars found yet*/ - - len_my_str = strlen(my_str); - - /* first check if all 'characters' of *my_str are digits, */ - /* str_ptr points to the first occurrence of a character */ - for (str_ptr=0; str_ptr<len_my_str; str_ptr++) { - if (str_ptr == 0) { /* there can be a '+' or a '-' in the first position */ - /* but I do not allow a negative value because there's */ - /* no negative amount of memory... */ - if (my_str[0] == '+') { - tempbuf[0] = my_str[0]; /* copy sign */ - /* this is ok but a digit MUST follow */ - str_ptr++; - /* If only a '+' was entered the next character is '\0'. */ - /* This is not a digit so the error message shows up */ - } - } - if (!isdigit(my_str[str_ptr])) { - break; - } - tempbuf[str_ptr] = my_str[str_ptr]; /* copy to return string */ - } /* endfor */ - - tempbuf[str_ptr] = 0x00; /* terminate return string, this was NOT done before this modification! */ - - if((my_str[str_ptr] == fir_c) || (my_str[str_ptr] == sec_c)) { - /*check which char it is, then set multiplier and indicate char found*/ - switch(my_str[str_ptr]) { - case 'K': - *multiplier = 'K'; - found++; - break; - case 'M': - *multiplier = 'M'; - found++; - break; - case 'G': - *multiplier = 'G'; - found++; - break; - } - /* my_str[str_ptr] = 0; */ /*change char in string to end of string char*/ - } - if (found == 0) { /*if input chars not found, indicate error*/ - return(1); } - else { - /* K, M or G should be the last character, something like 64M55 is invalid */ - if (str_ptr == len_my_str-1) { - return(0); /*if input chars found, return good status*/ - } else { - /* I only allow a 'B' to follow. This is not documented but reflects the */ - /* behaviour of earlier poe parsing. 64MB is valid, but after 'B' the */ - /* string must end */ - if (my_str[str_ptr+1] == 'B' && (str_ptr+1) == (len_my_str-1)) { - return(0); /*if input chars found, return good status*/ - } else { - return(1); - } /* endif */ - } /* endif */ - } -} -/* - ----------------------------------------------------------------- - Name: MPIDI_scan_str3() - - Function: Scan a flag string for 3 out of 3 possible - characters (K, M, G). Return a 1 if neither - character is found otherwise return the character - along with a buffer containing the string without - the character. - value are valid. If they are valid, the - multiplication of the number and the units - will be returned as an unsigned int. If the - number and units are invalid, a 1 will be returned. - - Description: search string for character or end of string - if string contains either entered character - check which char it is, set multiplier - if no chars found, return error - - Parameters: A0 = MPIDI_scan_str(A1, A2, A3, A4, A5, A6) - - A1 string to scan char * - A2 first char to scan for char - A3 second char to scan for char - A4 third char to scan for char - A5 multiplier char * - A6 returned string char * - - A0 Return Code int - - Return Codes: 0 OK - 1 input chars not found - ------------------------------------------------------------ -*/ -int MPIDI_scan_str3(char *my_str, char fir_c, char sec_c, char thr_c, char *multiplier, char *tempbuf) -{ - - int str_ptr; /*index counter into string*/ - int found; /*indicates whether one of input chars found*/ - int len_my_str; /*length of string with size and units*/ - - str_ptr = 0; /*start at beginning of string*/ - found = 0; /*no chars found yet*/ - - len_my_str = strlen(my_str); - - /* first check if all 'characters' of *my_str are digits, */ - /* str_ptr points to the first occurrence of a character */ - for (str_ptr=0; str_ptr<len_my_str; str_ptr++) { - if (str_ptr == 0) { /* there can be a '+' or a '-' in the first position */ - /* but I do not allow a negative value because there's */ - /* no negative amount of memory... */ - if (my_str[0] == '+') { - tempbuf[0] = my_str[0]; /* copy sign */ - /* this is ok but a digit MUST follow */ - str_ptr++; - /* If only a '+' was entered the next character is '\0'. */ - /* This is not a digit so the error message shows up */ - } - } - if (!isdigit(my_str[str_ptr])) { - break; - } - tempbuf[str_ptr] = my_str[str_ptr]; /* copy to return string */ - } /* endfor */ - - tempbuf[str_ptr] = 0x00; /* terminate return string, this was NOT done before this modification! */ - - if((my_str[str_ptr] == fir_c) || (my_str[str_ptr] == sec_c) || (my_str[str_ptr] == thr_c)) { - /*check which char it is, then set multiplier and indicate char found*/ - switch(my_str[str_ptr]) { - case 'K': - *multiplier = 'K'; - found++; - break; - case 'M': - *multiplier = 'M'; - found++; - break; - case 'G': - *multiplier = 'G'; - found++; - break; - } - /* my_str[str_ptr] = 0; */ /*change char in string to end of string char*/ - } - if (found == 0) { /*if input chars not found, indicate error*/ - return(1); } - else { - /* K, M or G should be the last character, something like 64M55 is invalid */ - if (str_ptr == len_my_str-1) { - return(0); /*if input chars found, return good status*/ - } else { - /* I only allow a 'B' to follow. This is not documented but reflects the */ - /* behaviour of earlier poe parsing. 64MB is valid, but after 'B' the */ - /* string must end */ - if (my_str[str_ptr+1] == 'B' && (str_ptr+1) == (len_my_str-1)) { - return(0); /*if input chars found, return good status*/ - } else { - return(1); - } /* endif */ - } /* endif */ - } -} - -/* - ----------------------------------------------------------------- - Name: MPIDI_checkit() - - Function: Determine whether a given number and units - value are valid. If they are valid, the - multiplication of the number and the units - will be returned as an unsigned int. If the - number and units are invalid, a 1 will be returned. - - Description: if units is G - if value is > 4 return error - else multiplier is 1G - else if units is M - if value is > 4K return error - else multiplier is 1M - else if units is K - if value is > 4M return error - else multiplier is 1K - if value < 1 return error - else - multiply value by multiplier - return result - - Parameters: A0 = MPIDI_checkit(A1, A2, A3) - - A1 given value int - A2 given units char * - A3 result unsigned int * - - A0 Return Code int - - Return Codes: 0 OK - 1 bad value - ------------------------------------------------------------ -*/ -int MPIDI_checkit(int myval, char myunits, unsigned int *mygoodval) -{ - int multiplier = ONE; /*units multiplier for entered value*/ - - if (myunits == 'G') { /*if units is G*/ - if (myval>4) return 1; /*entered value can't be greater than 4*/ - else multiplier = ONEG; /*if OK, mult value by units*/ - } - else if (myunits == 'M') { /*if units is M*/ - if (myval > (4*ONEK)) return 1; /*value can't be > 4096*/ - else multiplier = ONEM; /*if OK, mult value by units*/ - } - else if (myunits == 'K') { /*if units is K*/ - if (myval > (4*ONEM)) return 1; /*value can't be > 4M*/ - else multiplier = ONEK; /*if OK, mult value by units*/ - } - if (myval < 1) return 1; /*value can't be less than 1*/ - - *mygoodval = myval * multiplier; /*do multiplication*/ - return 0; /*good return*/ - -} - - - - /*************************************************************************** - Function Name: MPIDI_atoi - - Description: Convert a string into an interger. The string can be all - digits or includes symbols 'K', 'M'. - - Parameters: char * -- string to be converted - unsigned int * -- result val (caller to cast to int* or long*) - - Return: int 0 if AOK - number of errors. - ***************************************************************************/ -int MPIDI_atoi(char* str_in, unsigned int* val) -{ - char tempbuf[256]; - char size_mult; /* multiplier for size strings */ - int i, tempval; - int letter=0, retval=0; - - /***********************************/ - /* Check for letter */ - /***********************************/ - for (i=0; i<strlen(str_in); i++) { - if (!isdigit(str_in[i])) { - letter = 1; - break; - } - } - if (!letter) { /* only digits */ - errno = 0; /* should set errno to 0 before atoi() call */ - *val = atoi(str_in); - if (errno) { /* no check for negative integer, there's no '-' in the string */ - retval = errno; - } - } - else { - /***********************************/ - /* Check for K or M. */ - /***********************************/ - MPIDI_toupper(str_in); - retval = MPIDI_scan_str(str_in, 'M', 'K', &size_mult, tempbuf); - - if ( retval == 0) { - tempval = atoi(tempbuf); - - /***********************************/ - /* If 0 K or 0 M entered, set to 0 */ - /* otherwise, do conversion. */ - /***********************************/ - if (tempval != 0) - retval = MPIDI_checkit(tempval, size_mult, (unsigned int*)val); - else - *val = 0; - } - - if (retval == 0) { - tempval = atoi(tempbuf); - retval = MPIDI_checkit(tempval, size_mult, (unsigned int*)val); - } - else - *val = 0; - } - - return retval; -} - - /*************************************************************************** - Name: MPIDI_checkll() - - Function: Determine whether a given number and units - value are valid. If they are valid, the - multiplication of the number and the units - will be returned as an unsigned int. If the - number and units are invalid, a 1 will be returned. - - Description: if units is G - multiplier is 1G - else if units is M - multiplier is 1M - else if units is K - multiplier is 1K - else - return error - - multiply value by multiplier - return result - Parameters: A0 = MPIDI_checkll(A1, A2, A3) - - A1 given value int - A2 given units char * - A3 result long long * - - A0 Return Code int - - Return Codes: 0 OK - 1 bad value - ***************************************************************************/ - -int MPIDI_checkll(int myval, char myunits, long long *mygoodval) -{ - int multiplier; /* units multiplier for entered value */ - - if (myunits == 'G') { /* if units is G */ - multiplier = ONEG; - } - else if (myunits == 'M') { /* if units is M */ - multiplier = ONEM; - } - else if (myunits == 'K') { /* if units is K */ - multiplier = ONEK; - } - else - return 1; /* Unkonwn unit */ - - *mygoodval = (long long) myval * multiplier; /* do multiplication */ - return 0; /* good return */ -} - - -int MPIDI_atoll(char* str_in, long long* val) -{ - char tempbuf[256]; - char size_mult; /* multiplier for size strings */ - int i, tempval; - int letter=0, retval=0; - - /*****************************************/ - /* Check for letter, if none, MPIDI_atoi */ - /*****************************************/ - for (i=0; i<strlen(str_in); i++) { - if (!isdigit(str_in[i])) { - letter = 1; - break; - } - } - if (!letter) { /* only digits */ - errno = 0; - *val = atoll(str_in); - if (errno) { - retval = errno; - } - } - else { - /***********************************/ - /* Check for K or M. */ - /***********************************/ - MPIDI_toupper(str_in); - retval= MPIDI_scan_str3(str_in, 'G', 'M', 'K', &size_mult, tempbuf); - - if ( retval == 0) { - tempval = atoi(tempbuf); - - /***********************************/ - /* If 0 K or 0 M entered, set to 0 */ - /* otherwise, do conversion. */ - /***********************************/ - if (tempval != 0) - retval = MPIDI_checkll(tempval, size_mult, val); - else - *val = 0; - } - } - - return retval; -} - - - -/****************************************************/ -/* Collective Selection Benchmarking utils */ -/****************************************************/ - -const char * xfer_array_str[PAMI_XFER_COUNT]; -int task_id; -int num_tasks; - -static int MPIDI_collsel_print_usage() -{ - if(!task_id) - fputs("Usage: pami_tune (MPICH) [options]\n\ -Options:\n\ - -c Comma separated list of collectives to benchmark\n\ - Valid options are: \n\ - allgather, allgatherv_int, allreduce, alltoall,\n\ - alltoallv_int, ambroadcast, amgather, amreduce,\n\ - amscatter, barrier, broadcast, gather, gatherv_int,\n\ - reduce, reduce_scatter, scan, scatter, scatterv_int\n\ - (Default: all collectives)\n\n\ - -m Comma separated list of message sizes to benchmark\n\ - (Default: 1 to 2^k, where k <= 20)\n\n\ - -g Comma separated list of geometry sizes to benchmark\n\ - (Default: Powers of 2 (plus and minus one as well))\n\n\ - -i Number of benchmark iterations per algorithm\n\ - (Default: 1000. Maximum is 2000)\n\n\ - -f <file> Input INI file containing benchmark parameters\n\ - You can override a parameter with a command line argument\n\n\ - -o <file> Output XML file containing benchmark results\n\ - (Default: pami_tune_results.xml)\n\n\ - -d Diagnostics mode. Verify correctness of collective algorithms\n\ - (Default: Disabled)\n\n\ - -p Print the Collective Algorithm performance data to standard output\n\ - (Default: Disabled)\n\n\ - -h Print this help message\n\n", stdout); - - return 0; -} - - -static void MPIDI_collsel_init_advisor_params(advisor_params_t *params) -{ - params->num_collectives = 0; - params->num_procs_per_node = 0; - params->num_geometry_sizes = 0; - params->num_message_sizes = 0; - params->collectives = NULL; - params->procs_per_node = NULL; - params->geometry_sizes = NULL; - params->message_sizes = NULL; - /* Set the following to -1, so that we can - check if the user has set them or not */ - params->iter = -1; - params->verify = -1; - params->verbose = -1; - params->checkpoint = -1; -} - - -static void MPIDI_collsel_free_advisor_params(advisor_params_t *params) -{ - if(params->collectives) MPL_free(params->collectives); - if(params->procs_per_node) MPL_free(params->procs_per_node); - if(params->geometry_sizes) MPL_free(params->geometry_sizes); - if(params->message_sizes) MPL_free(params->message_sizes); -} - - -static void MPIDI_collsel_init_xfer_tables() -{ - xfer_array_str[PAMI_XFER_BROADCAST] ="broadcast"; - xfer_array_str[PAMI_XFER_ALLREDUCE] ="allreduce"; - xfer_array_str[PAMI_XFER_REDUCE] ="reduce"; - xfer_array_str[PAMI_XFER_ALLGATHER] ="allgather"; - xfer_array_str[PAMI_XFER_ALLGATHERV] ="allgatherv"; - xfer_array_str[PAMI_XFER_ALLGATHERV_INT]="allgatherv_int"; - xfer_array_str[PAMI_XFER_SCATTER] ="scatter"; - xfer_array_str[PAMI_XFER_SCATTERV] ="scatterv"; - xfer_array_str[PAMI_XFER_SCATTERV_INT] ="scatterv_int"; - xfer_array_str[PAMI_XFER_GATHER] ="gather"; - xfer_array_str[PAMI_XFER_GATHERV] ="gatherv"; - xfer_array_str[PAMI_XFER_GATHERV_INT] ="gatherv_int"; - xfer_array_str[PAMI_XFER_BARRIER] ="barrier"; - xfer_array_str[PAMI_XFER_ALLTOALL] ="alltoall"; - xfer_array_str[PAMI_XFER_ALLTOALLV] ="alltoallv"; - xfer_array_str[PAMI_XFER_ALLTOALLV_INT] ="alltoallv_int"; - xfer_array_str[PAMI_XFER_SCAN] ="scan"; - xfer_array_str[PAMI_XFER_REDUCE_SCATTER]="reduce_scatter"; - xfer_array_str[PAMI_XFER_AMBROADCAST] ="ambroadcast"; - xfer_array_str[PAMI_XFER_AMSCATTER] ="amscatter"; - xfer_array_str[PAMI_XFER_AMGATHER] ="amgather"; - xfer_array_str[PAMI_XFER_AMREDUCE] ="amreduce"; -} - -static int MPIDI_collsel_process_collectives(char *coll_arg, advisor_params_t *params) -{ - int i, ret = 0, arg_len = strlen(coll_arg); - char *collectives = (char *) MPL_malloc(arg_len + 1); - char *coll; - int infolevel = 0; -#if (MPIDI_STATISTICS || MPIDI_PRINTENV) - if(MPIDI_Process.mp_infolevel >= 1) - infolevel = 1; -#endif - /* if already set via config file, free it */ - if(params->collectives) - { - MPL_free(params->collectives); - params->num_collectives = 0; - } - /* Allocating some extra space should be fine */ - params->collectives = (pami_xfer_type_t *)MPL_malloc(sizeof(pami_xfer_type_t)*PAMI_XFER_COUNT); - - strcpy(collectives, coll_arg); - coll = strtok(collectives,","); - while (coll != NULL) - { - int invalid_collective = 0; - for(i=0; i<PAMI_XFER_COUNT; i++) - { - if(strcmp(coll, xfer_array_str[i]) == 0) - { - if(i == 4) - { - if(infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (pami_tune) doesn't support tuning for ALLGATHERV. ALLGATHERV tuning will be skipped.\nTune for ALLGATHERV_INT instead\n"); - invalid_collective = 1; - break; - } - else if(i == 7) - { - if(infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (pami_tune) doesn't support tuning for SCATTERV. SCATTERV tuning will be skipped.\nTune for SCATTERV_INT instead\n"); - invalid_collective = 1; - break; - } - else if(i == 10) - { - if(infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (pami_tune) doesn't support tuning for GATHERV. GATHERV tuning will be skipped.\nTune for GATHERV_INT instead\n"); - invalid_collective = 1; - break; - } - else if(i == 14) - { - if(infolevel >= 1) - fprintf(stderr,"WARNING: MPICH (pami_tune) doesn't support tuning for ALLTOALLV. ALLTOALLV tuning will be skipped.\nTune for ALLTOALLV_INT instead\n"); - invalid_collective = 1; - break; - } - else - { - params->collectives[params->num_collectives++] = i; - break; - } - } - } - /* arg did not match any collective */ - if(i == PAMI_XFER_COUNT || invalid_collective) - { - if(!task_id) - { - fprintf(stderr, "Invalid collective: %s\n", coll); - } - break; - } - coll = strtok(NULL,","); - } - if(params->num_collectives == 0) - { - MPL_free(params->collectives); - params->collectives = NULL; - ret = 1; - } - - MPL_free(collectives); - return ret; -} - - -static int MPIDI_collsel_process_msg_sizes(char *msg_sizes_arg, advisor_params_t *params) -{ - int ret = 0, arg_len = strlen(msg_sizes_arg); - char *msg_sizes = (char *) MPL_malloc(arg_len + 1); - char *msg_sz; - size_t tmp; - size_t array_size = 50; - /* if already set via config file, free it */ - if(params->message_sizes) - { - MPL_free(params->message_sizes); - params->num_message_sizes = 0; - } - /* Allocating some extra space should be fine */ - params->message_sizes = (size_t *)MPL_malloc(sizeof(size_t) * array_size); - - strcpy(msg_sizes, msg_sizes_arg); - msg_sz = strtok(msg_sizes,","); - while (msg_sz != NULL) - { - tmp = strtol(msg_sz, NULL, 10); - if(tmp == 0) - { - MPL_free(params->message_sizes); - params->message_sizes = NULL; - if(!task_id) - { - fprintf(stderr, "Invalid message size: %s\n", msg_sz); - } - ret = 1; - break; - } - - if(params->num_message_sizes >= array_size) - { - array_size *= 2; - params->message_sizes = (size_t *) MPL_realloc(params->message_sizes, - sizeof(size_t) * array_size); - } - params->message_sizes[params->num_message_sizes++] = tmp; - msg_sz = strtok(NULL,","); - } - MPL_free(msg_sizes); - return ret; -} - - -static int MPIDI_collsel_process_geo_sizes(char *geo_sizes_arg, advisor_params_t *params) -{ - int ret = 0, arg_len = strlen(geo_sizes_arg); - char *geo_sizes = (char *) MPL_malloc(arg_len + 1); - char *geo_sz; - size_t tmp; - size_t array_size = 50; - /* if already set via config file, free it */ - if(params->geometry_sizes) - { - MPL_free(params->geometry_sizes); - params->num_geometry_sizes = 0; - } - /* Allocating some extra space should be fine */ - params->geometry_sizes = (size_t *)MPL_malloc(sizeof(size_t) * array_size); - - strcpy(geo_sizes, geo_sizes_arg); - geo_sz = strtok(geo_sizes,","); - while (geo_sz != NULL) - { - tmp = strtol(geo_sz, NULL, 10); - if(tmp < 2 || tmp > num_tasks) - { - MPL_free(params->geometry_sizes); - params->geometry_sizes = NULL; - if(!task_id) - { - fprintf(stderr, "Invalid geometry size: %s\n", geo_sz); - } - ret = 1; - break; - } - - if(params->num_geometry_sizes >= array_size) - { - array_size *= 2; - params->geometry_sizes = (size_t *) MPL_realloc(params->geometry_sizes, - sizeof(size_t) * array_size); - } - params->geometry_sizes[params->num_geometry_sizes++] = tmp; - geo_sz = strtok(NULL,","); - } - MPL_free(geo_sizes); - return ret; -} - - -static int MPIDI_collsel_process_output_file(char *filename, char **out_file) -{ - char *newname; - int i, filename_len, ret = 0; - struct stat ost; - - filename_len = strlen(filename); - - /* Check if file already exists */ - if(stat(filename, &ost) == 0) - { - if(!S_ISREG(ost.st_mode)) - { - fprintf(stderr, "Error: %s exists and is not a regular file\n", filename); - return 1; - } - fprintf(stderr, "File %s already exists, renaming existing file\n", filename); - - newname = (char *) MPL_malloc(filename_len + 5); - for (i = 0; i < 500; ++i) - { - sprintf(newname,"%s.%d", filename, i); - if(!(access(newname, F_OK) == 0)) - { - ret = rename(filename, newname); - break; - } - } - MPL_free(newname); - if(i == 500 || ret != 0) - { - fprintf(stderr, "Error renaming file\n"); - return 1; - } - } - /* if file name is already set via config file, free it */ - if(*out_file) MPL_free(*out_file); - *out_file = (char *)MPL_malloc(filename_len + 1); - strcpy(*out_file, filename); - - return ret; -} - - -static char* MPIDI_collsel_ltrim(char *line) -{ - while(*line && isspace(*line)) - ++line; - - return line; -} - -static char* MPIDI_collsel_rtrim(char *line) -{ - char *end = line + strlen(line); - while(end > line && isspace(*--end)) - *end = '\0'; - - return line; -} - -static int MPIDI_collsel_checkvalue(char *name, char *value, const char *filename, int *ival) -{ - int ret = 0; - char *tmp; - *ival = (int)strtol(value, &tmp, 10); - if(*ival != 1) - { - if((*ival == 0 && value == tmp)|| *ival != 0) - { - if(!task_id) - fprintf(stderr, "Invalid value for %s parameter: %s in file: %s\n", name, value, filename); - ret = 1; - } - } - return ret; -} - - -static int MPIDI_collsel_process_ini_file(const char *filename, advisor_params_t *params, char **out_file) -{ - char *line, *start, *name, *value; - int ret = 0; - struct stat ist; - - if(stat(filename, &ist) == 0) - { - if(!S_ISREG(ist.st_mode)) - { - fprintf(stderr, "Error: %s is not a regular file\n", filename); - return 1; - } - } - - FILE *file = fopen(filename, "r"); - if(!file) - { - fprintf(stderr, "Error: Cannot open file %s: %s\n", - filename, strerror(errno)); - return 1; - } - line = (char *) MPL_malloc(2000); - - while (fgets(line, 2000, file) != NULL) - { - start = MPIDI_collsel_ltrim(MPIDI_collsel_rtrim(line)); - /* Skip comments and sections */ - if(*start == ';' || *start == '[' || *start == '#') - continue; - name = strtok(line, "="); - if(name == NULL) continue; - value = strtok(NULL, "="); - if(value == NULL) continue; - name = MPIDI_collsel_rtrim(name); - value = MPIDI_collsel_ltrim(value); - /* Do not override command line values if they are set */ - if(strcmp(name, "collectives") == 0) - { - if(!params->collectives) - ret = MPIDI_collsel_process_collectives(value, params); - } - else if(strcmp(name, "message_sizes") == 0) - { - if(!params->message_sizes) - ret = MPIDI_collsel_process_msg_sizes(value, params); - } - else if(strcmp(name, "geometry_sizes") == 0) - { - if(!params->geometry_sizes) - ret = MPIDI_collsel_process_geo_sizes(value, params); - } - else if(strcmp(name, "output_file") == 0) - { - if(!*out_file && !task_id) /* Only task 0 creates o/p file */ - ret = MPIDI_collsel_process_output_file(value, out_file); - } - else if(strcmp(name, "iterations") == 0) - { - if(params->iter == -1) - { - params->iter = atoi(value); - if(params->iter <= 0 || params->iter > PAMI_TUNE_MAX_ITER) - { - if(!task_id) - fprintf(stderr, "Invalid iteration count: %s in file: %s\n", value, filename); - ret = 1; - } - } - } - else if(strcmp(name, "verbose") == 0) - { - if(params->verbose == -1) - ret = MPIDI_collsel_checkvalue(name, value, filename, ¶ms->verbose); - } - else if(strcmp(name, "diagnostics") == 0) - { - if(params->verify == -1) - ret = MPIDI_collsel_checkvalue(name, value, filename, ¶ms->verify); - } - /*else if(strcmp(name, "checkpoint") == 0) - { - if(params->checkpoint == -1) - ret = MPIDI_collsel_checkvalue(name, value, filename, ¶ms->checkpoint); - }*/ - else - { - fprintf(stderr, "Invalid parameter: %s in file: %s\n", name, filename); - ret = 1; - } - if(ret) break; - } - MPL_free(line); - fclose(file); - - return ret; -} - - -static int MPIDI_collsel_process_arg(int argc, char *argv[], advisor_params_t *params, char ** out_file) -{ - int i,c,ret = 0; - char *in_file = NULL; - - MPIDI_collsel_init_xfer_tables(); - params->verify = 0; - - opterr = 0; - while ((c = getopt (argc, argv, "c:m:g:f:o:i:p::d::x::h::")) != -1) - { - switch (c) - { - case 'c': - ret = MPIDI_collsel_process_collectives(optarg, params); - break; - case 'm': - ret = MPIDI_collsel_process_msg_sizes(optarg, params); - break; - case 'g': - ret = MPIDI_collsel_process_geo_sizes(optarg, params); - break; - case 'f': - in_file = (char *) optarg; - break; - case 'o': - if(!task_id) /* Only task 0 creates o/p file */ - ret = MPIDI_collsel_process_output_file(optarg, out_file); - break; - case 'i': - params->iter = atoi(optarg); - if(params->iter <= 0 || params->iter > PAMI_TUNE_MAX_ITER) - { - if(!task_id) - fprintf(stderr, "Invalid iteration count %s\n", optarg); - ret = 1; - } - break; - case 'd': - params->verify = 1; - break; - case 'p': - params->verbose = 1; - break; - case 'x': - /*params->checkpoint = 1;*/ - break; - case 'h': - ret = 2; - break; - case '?': - if(!task_id) - { - if (optopt == 'c' || optopt == 'm' || optopt == 'g' || - optopt == 'f' || optopt == 'o' || optopt == 'i') - { - fprintf (stderr, "Option -%c requires an argument.\n", optopt); - } - else if (isprint (optopt)) - { - fprintf (stderr, "Unknown option `-%c'.\n", optopt); - } - else - { - fprintf (stderr, "Unknown option character `\\x%x'.\n", optopt); - } - } - ret = 1; - break; - default: - abort(); - break; - } - if(ret) return ret; - } - /* INI file is processed last as we don't wan't to override - * command line parameters. Also any invalid value in the INI - * file if overriden by a command line flag should not generate - * an error - */ - if(in_file != NULL) - { - ret = MPIDI_collsel_process_ini_file(in_file, params, out_file); - if(ret) return ret; - } - - if(!task_id) - { - if (optind < argc) - { - printf ("Non-option arguments: "); - while (optind < argc) - printf ("%s ", argv[optind++]); - printf ("\n"); - } - } - - /* If user did not specify any collectives, benchmark all */ - if(params->num_collectives == 0) - { - params->collectives = (pami_xfer_type_t *)MPL_malloc(sizeof(pami_xfer_type_t)*PAMI_XFER_COUNT); - for(i = 0; i < PAMI_XFER_COUNT; i++) - { - if(i == 4 || i == 7 || i == 10 || i == 14) - i++; - params->collectives[params->num_collectives++] = i; - } - } - /* If user did not set any of the following parameters, set default value */ - if(params->iter == -1) params->iter = 1000; - if(params->verbose == -1) params->verbose = 0; - if(params->verify == -1) params->verify = 0; - if(params->checkpoint == -1) params->checkpoint = 0; - return 0; -} - - -static void MPIDI_collsel_print_params(advisor_params_t *params, char *output_file) -{ - size_t i; - printf(" Benchmark Parameters\n"); - printf(" --------------------\n"); - - printf(" Collectives:\n "); - for(i=0; i<params->num_collectives; i++){ - printf(" %s", xfer_array_str[params->collectives[i]]); - } - printf("\n Geometry sizes:\n "); - for(i=0; i<params->num_geometry_sizes; i++){ - printf(" %zu", params->geometry_sizes[i]); - } - printf("\n Message sizes:\n "); - for(i=0; i<params->num_message_sizes; i++){ - printf(" %zu", params->message_sizes[i]); - } - printf("\n Iterations : %d\n", params->iter); - printf(" Output file : %s\n", output_file); -/* printf(" Checkpoint mode : %d\n", params->checkpoint); */ - printf(" Diagnostics mode : %d\n", params->verify); - printf(" Verbose mode : %d\n", params->verbose); -} - - -int MPIDI_collsel_pami_tune_parse_params(int argc, char ** argv) -{ - MPIDI_Collsel_output_file = NULL; - num_tasks = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_NUM_TASKS).value.intval; - task_id = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval; - MPIDI_collsel_init_advisor_params(&MPIDI_Collsel_advisor_params); - if(num_tasks < 2) - { - fprintf (stderr, "Error: pami_tune must be run with more than one task\n"); - fflush(stderr); - MPIDI_collsel_print_usage(); - return PAMI_ERROR; - } - MPIDI_Collsel_advisor_params.procs_per_node = (size_t*)MPL_malloc(1 * sizeof(size_t)); - MPIDI_Collsel_advisor_params.num_procs_per_node = 1; - MPIDI_Collsel_advisor_params.procs_per_node[0] = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_NUM_LOCAL_TASKS ).value.intval; - - int result = MPIDI_collsel_process_arg(argc, argv, &MPIDI_Collsel_advisor_params, &MPIDI_Collsel_output_file); - if(result) - { - MPIDI_collsel_print_usage(); - return PAMI_ERROR; - } - - if(MPIDI_Collsel_output_file == NULL && !task_id) - { - if(MPIDI_collsel_process_output_file("pami_tune_results.xml", &MPIDI_Collsel_output_file)) - { - return PAMI_ERROR; - } - } - if(MPIDI_Collsel_advisor_params.verbose && !task_id) - MPIDI_collsel_print_params(&MPIDI_Collsel_advisor_params, MPIDI_Collsel_output_file); - - return PAMI_SUCCESS; - -} - -void MPIDI_collsel_pami_tune_cleanup() -{ - MPIDI_collsel_free_advisor_params(&MPIDI_Collsel_advisor_params); -} - - - -/**********************************************************/ -/* CUDA Utilities */ -/**********************************************************/ -#if CUDA_AWARE_SUPPORT -int CudaMemcpy(void* dst, const void* src, size_t count, int kind) -{ - return (*pamidCudaMemcpy)(dst, src, count, kind); -} -int CudaPointerGetAttributes(struct cudaPointerAttributes* attributes, const void* ptr) -{ - return (*pamidCudaPointerGetAttributes)(attributes, ptr); -} -const char* CudaGetErrorString( int error) -{ - return (*pamidCudaGetErrorString)(error); -} -#endif - -inline bool MPIDI_enable_cuda() -{ - bool result = false; -#if CUDA_AWARE_SUPPORT - pamidCudaPtr = dlopen("libcudart.so", RTLD_NOW|RTLD_GLOBAL); - if(pamidCudaPtr == NULL) - { - TRACE_ERR("failed to open libcudart.so error=%s\n", dlerror()); - return result; - } - else - { - pamidCudaMemcpy = (int (*)())dlsym(pamidCudaPtr, "cudaMemcpy"); - if(pamidCudaMemcpy == NULL) - { - dlclose(pamidCudaPtr); - return result; - } - pamidCudaPointerGetAttributes = (int (*)())dlsym(pamidCudaPtr, "cudaPointerGetAttributes"); - if(pamidCudaPointerGetAttributes == NULL) - { - dlclose(pamidCudaPtr); - return result; - } - pamidCudaGetErrorString = (const char* (*)())dlsym(pamidCudaPtr, "cudaGetErrorString"); - if(pamidCudaGetErrorString == NULL) - { - dlclose(pamidCudaPtr); - return result; - } - result = true; - } -#endif - return result; -} - -inline bool MPIDI_cuda_is_device_buf(const void* ptr) -{ - bool result = false; -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on) - { - if(ptr != MPI_IN_PLACE) - { - struct cudaPointerAttributes cuda_attr; - cudaError_t e= CudaPointerGetAttributes ( & cuda_attr, ptr); - - if (e != cudaSuccess) - result = false; - else if (cuda_attr.memoryType == cudaMemoryTypeDevice) - result = true; - else - result = false; - } - } -#endif - return result; -} - - -/**********************************************************/ -/* End CUDA Utilities */ -/**********************************************************/ - -#if defined(MPID_USE_NODE_IDS) -#undef FUNCNAME -#define FUNCNAME MPID_Get_node_id -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -static int _g_max_node_id = -1; -int MPID_Get_node_id(MPIR_Comm *comm, int rank, MPID_Node_id_t *id_p) -{ - int mpi_errno = MPI_SUCCESS; - uint32_t node_id; - uint32_t offset; - uint32_t max_nodes; - - if(!PAMIX_Extensions.is_local_task.node_info) - { - *id_p = rank; - } - else - { - pami_result_t rc = PAMIX_Extensions.is_local_task.node_info(comm->vcr[rank]->taskid, - &node_id,&offset,&max_nodes); - if(rc != PAMI_SUCCESS) - *id_p = rank; - else - *id_p = node_id; - } - - fn_fail: - return mpi_errno; -} - -#undef FUNCNAME -#define FUNCNAME MPID_Get_max_node_id -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_Get_max_node_id(MPIR_Comm *comm, MPID_Node_id_t *max_id_p) -{ - int mpi_errno = MPI_SUCCESS; - uint32_t node_id; - uint32_t offset; - uint32_t max_nodes; - if(_g_max_node_id == -1) - { - if(!PAMIX_Extensions.is_local_task.node_info) - MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**notimpl"); - - pami_result_t rc = PAMIX_Extensions.is_local_task.node_info(comm->vcr[0]->taskid, - &node_id,&offset,&max_nodes); - if(rc != PAMI_SUCCESS) MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**notimpl"); - *max_id_p = max_nodes; - _g_max_node_id = max_nodes; - } - else - *max_id_p = _g_max_node_id; - - fn_fail: - return mpi_errno; -} -#endif -#undef FUNCNAME diff --git a/src/mpid/pamid/src/mpix/Makefile.mk b/src/mpid/pamid/src/mpix/Makefile.mk deleted file mode 100644 index 8f4b990..0000000 --- a/src/mpid/pamid/src/mpix/Makefile.mk +++ /dev/null @@ -1,29 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/mpix/mpix.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/mpix/mpix.c b/src/mpid/pamid/src/mpix/mpix.c deleted file mode 100644 index 22955fd..0000000 --- a/src/mpid/pamid/src/mpix/mpix.c +++ /dev/null @@ -1,1210 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/mpix/mpix.c - * \brief This file is for MPI extensions that MPI apps are likely to use. - */ - -#include <mpidimpl.h> - -#ifdef __BGQ__ -#include <stdlib.h> -#include <spi/include/kernel/location.h> -#endif /* __BGQ__ */ - -MPIX_Hardware_t MPIDI_HW; - -/* Determine the number of torus dimensions. Implemented to keep this code - * architecture-neutral. do we already have this cached somewhere? - * this call is non-destructive (no asserts), however future calls are - * destructive so the user needs to verify numdimensions != -1 - */ - - -static void -MPIX_Init_hw(MPIX_Hardware_t *hw) -{ - memset(hw, 0, sizeof(MPIX_Hardware_t)); - - hw->clockMHz = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_CLOCK_MHZ).value.intval; - hw->memSize = PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_MEM_SIZE).value.intval; - -#ifdef __BGQ__ - int i=0; - const pamix_torus_info_t *info = PAMIX_Torus_info(); - /* The extension returns a "T" dimension */ - hw->torus_dimension = info->dims-1; - - hw->psize = info->size[info->dims-1]; - for(i=0;i<info->dims-1;i++) - { - hw->Size[i] = info->size[i]; - hw->Coords[i] = info->coord[i]; - hw->isTorus[i] = info->torus[i]; - hw->psize = hw->psize * info->size[i]; - } - - hw->prank = info->coord[0]; - for(i=1;i<info->dims;i++) - { - hw->prank = (hw->prank * info->size[i] + info->coord[i]); - } - - /* The torus extension returns "T" as the last element */ - hw->coreID = info->coord[info->dims-1]; - hw->ppn = info->size[info->dims-1]; -#endif -} - - -void -MPIX_Init() -{ - MPIX_Init_hw(&MPIDI_HW); -} - - -#if !defined(__AIX__) -extern int backtrace(void **buffer, int size); /**< GlibC backtrace support */ -extern char **backtrace_symbols(void *const *buffer, int size); /**< GlibC backtrace support */ -void MPIX_Dump_stacks() -{ - const size_t SIZE=32; - void *array[SIZE]; - size_t i; - size_t size = backtrace(array, SIZE); - char **bt_strings = backtrace_symbols(array, size); - fprintf(stderr, "Dumping %zd frames:\n", size - 1); - for (i = 1; i < size; i++) - { - if (bt_strings != NULL) - fprintf(stderr, "\tFrame %zd: %p: %s\n", i, array[i], bt_strings[i]); - else - fprintf(stderr, "\tFrame %zd: %p\n", i, array[i]); - } - /* This is #ifdef'd out: In debug libraries, it causes a compile error. */ -#if 0 - free(bt_strings); /* Since this is not allocated by MPL_malloc, do not use MPL_free */ -#endif -} -#else -void MPIX_Dump_stacks(){} -#endif - - -void -MPIX_Progress_poke() -{ - MPID_Progress_poke(); -} - - -int -MPIX_Progress_quiesce(double timeout) -{ - int rc; - timeout *= 1.0e6; /* convert to uSec */ - unsigned long long cycles = timeout; /* convert to long long */ - /* default to 10mS */ - cycles = (cycles ? cycles : 10000) * MPIDI_HW.clockMHz; - unsigned long long t0; - t0 = PAMI_Wtimebase(MPIDI_Client); - while (PAMI_Wtimebase(MPIDI_Client) - t0 < cycles) { - rc = MPID_Progress_wait_inline(1); - if (rc != MPI_SUCCESS) return rc; - } - return MPI_SUCCESS; -} - - -int -MPIX_Comm_rank2global(MPI_Comm comm, int crank, int *grank) -{ - if (grank == NULL) - return MPI_ERR_ARG; - - MPIR_Comm *comm_ptr = NULL; - MPIR_Comm_get_ptr(comm, comm_ptr); - if (comm_ptr == NULL) - return MPI_ERR_COMM; - - if (crank >= comm_ptr->local_size) - return MPI_ERR_RANK; - - *grank = MPID_VCR_GET_LPID(comm_ptr->vcr, crank); - return MPI_SUCCESS; -} - - -int -MPIX_Hardware(MPIX_Hardware_t *hw) -{ - if (hw == NULL) - return MPI_ERR_ARG; - - /* - * We've already initialized the hw structure in MPID_Init, - * so just copy it to the users buffer - */ - memcpy(hw, &MPIDI_HW, sizeof(MPIX_Hardware_t)); - return MPI_SUCCESS; -} - -#if (MPIDI_PRINTENV || MPIDI_STATISTICS || MPIDI_BANNER) -void mpc_statistics_write() __attribute__ ((alias("MPIX_statistics_write"))); -void mp_statistics_write() __attribute__ ((alias("MPIXF_statistics_write"))); -void mp_statistics_write_() __attribute__ ((alias("MPIXF_statistics_write"))); -void mp_statistics_write__() __attribute__ ((alias("MPIXF_statistics_write"))); -void mpc_statistics_zero() __attribute__ ((alias("MPIX_statistics_zero"))); -void mp_statistics_zero() __attribute__ ((alias("MPIXF_statistics_zero"))); -void mp_statistics_zero_() __attribute__ ((alias("MPIXF_statistics_zero"))); -void mp_statistics_zero__() __attribute__ ((alias("MPIXF_statistics_zero"))); - - /* ------------------------------------------- */ - /* - MPIDI_Statistics_zero and -------- */ - /* - MPIDI_Statistics_write can be -------- */ - /* - called during init and finalize -------- */ - /* - PE utiliti routines -------- */ - /* -------------------------------------------- */ - -int -MPIDI_Statistics_zero(void) -{ - int rc=0; - - mpid_statp->sends = 0; - mpid_statp->sendsComplete = 0; - mpid_statp->sendWaitsComplete = 0; - mpid_statp->recvs = 0; - mpid_statp->recvWaitsComplete = 0; - mpid_statp->earlyArrivals = 0; - mpid_statp->earlyArrivalsMatched = 0; - mpid_statp->lateArrivals = 0; - mpid_statp->unorderedMsgs = 0; - - return (rc); /* to map with current PE support */ -} - /*************************************************************************** - Function Name: _MPIX_statistics_zero - - Description: Call the corresponding MPIDI_statistics_zero function to initialize/clear - statistics counter. - - Parameters: - Name Type I/O - void - int >0 Success - <0 statistics not enable - ***************************************************************************/ - -int _MPIX_statistics_zero (void) -{ - int rc = MPIDI_Statistics_zero(); - if (rc < 0) { - MPID_assert(rc == PAMI_SUCCESS); - } - return(rc); -} - -int MPIX_statistics_zero(void) -{ - return(_MPIX_statistics_zero()); -} - -void MPIXF_statistics_zero(int *rc) -{ - *rc = _MPIX_statistics_zero(); -} - - -int -MPIDI_Statistics_write(FILE *statfile) { - - int rc=-1; - int i; - char time_buf[201]; - extern pami_extension_t extension; - pami_configuration_t query_stat; - pami_statistics_t *pami_stat; - pami_counter_t *pami_counters; - long long Tot_dup_pkt_cnt=0; - long long Tot_retrans_pkt_cnt=0; - long long Tot_gho_pkt_cnt=0; - long long Tot_pkt_sent_cnt=0; - long long Tot_pkt_recv_cnt=0; - long long Tot_data_sent=0; - long long Tot_data_recv=0; - extern long mem_hwmark; - - memset(&time_buf,0, 201); - sprintf(time_buf, __DATE__" "__TIME__); - mpid_statp->buffer_mem_hwmark = mem_hwmark; - mpid_statp->sendWaitsComplete = mpid_statp->sends - mpid_statp->sendsComplete; - fprintf(statfile,"Start of task (pid=%d) statistics at %s \n", getpid(), time_buf); - fprintf(statfile, "MPICH: sends = %ld\n", mpid_statp->sends); - fprintf(statfile, "MPICH: sendsComplete = %ld\n", mpid_statp->sendsComplete); - fprintf(statfile, "MPICH: sendWaitsComplete = %ld\n", mpid_statp->sendWaitsComplete); - fprintf(statfile, "MPICH: recvs = %ld\n", mpid_statp->recvs); - fprintf(statfile, "MPICH: recvWaitsComplete = %ld\n", mpid_statp->recvWaitsComplete); - fprintf(statfile, "MPICH: earlyArrivals = %ld\n", mpid_statp->earlyArrivals); - fprintf(statfile, "MPICH: earlyArrivalsMatched = %ld\n", mpid_statp->earlyArrivalsMatched); - fprintf(statfile, "MPICH: lateArrivals = %ld\n", mpid_statp->lateArrivals); - fprintf(statfile, "MPICH: unorderedMsgs = %ld\n", mpid_statp->unorderedMsgs); - fprintf(statfile, "MPICH: buffer_mem_hwmark = %ld\n", mpid_statp->buffer_mem_hwmark); - fflush(statfile); - memset(&query_stat,0, sizeof(query_stat)); - query_stat.name = (pami_attribute_name_t)PAMI_CONTEXT_STATISTICS ; - rc = PAMI_Context_query(MPIDI_Context[0], &query_stat, 1); - pami_stat = (pami_statistics_t*)query_stat.value.chararray; - pami_counters = pami_stat->counters; - if (!rc) { - for (i = 0; i < pami_stat->count; i ++) { - printf("+++%s:%llu\n", pami_counters[i].name, pami_counters[i].value); - if (!strncasecmp("Duplicate Pkt Count",pami_counters[i].name,19)) { - Tot_dup_pkt_cnt=pami_counters[i].value; - } else if (!strncasecmp("Retransmit Pkt Count",pami_counters[i].name,20)) { - Tot_retrans_pkt_cnt=pami_counters[i].value; - } else if (!strncasecmp("Ghost Pkt Count",pami_counters[i].name,15)) { - Tot_gho_pkt_cnt=pami_counters[i].value; - } else if (!strncasecmp("Packets Sent",pami_counters[i].name,12) && - (!(strchr(pami_counters[i].name, 'v')))) { - /* Packets Sent, not Packets Sent via SHM */ - Tot_pkt_sent_cnt=pami_counters[i].value; - } else if (!strncasecmp("Packets Received",pami_counters[i].name,16) && - (!(strchr(pami_counters[i].name, 'S')))) { - /* Packets Received, not Packets Received via SHM */ - Tot_pkt_recv_cnt=pami_counters[i].value; - } else if (!strncasecmp("Data Sent",pami_counters[i].name,9) && - (!(strchr(pami_counters[i].name, 'v')))) { - /* Data Sent, not Data Sent via SHM */ - Tot_data_sent=pami_counters[i].value; - } else if (!strncasecmp("Data Received",pami_counters[i].name,13) && - (!(strchr(pami_counters[i].name, 'S')))) { - /* Data Received, not Data Received via SHM */ - Tot_data_recv=pami_counters[i].value; - } - } - fprintf(statfile, "PAMI: Tot_dup_pkt_cnt=%lld\n", Tot_dup_pkt_cnt); - fprintf(statfile, "PAMI: Tot_retrans_pkt_cnt=%lld\n", Tot_retrans_pkt_cnt); - fprintf(statfile, "PAMI: Tot_gho_pkt_cnt=%lld\n", Tot_gho_pkt_cnt); - fprintf(statfile, "PAMI: Tot_pkt_sent_cnt=%lld\n", Tot_pkt_sent_cnt); - fprintf(statfile, "PAMI: Tot_pkt_recv_cnt=%lld\n", Tot_pkt_recv_cnt); - fprintf(statfile, "PAMI: Tot_data_sent=%lld\n", Tot_data_sent); - fprintf(statfile, "PAMI: Tot_data_recv=%lld\n", Tot_data_recv); - fflush(statfile); - } else { - TRACE_ERR("PAMID: PAMI_Context_query() with PAMI_CONTEXT_STATISTICS failed rc =%d\ -n",rc); - } - return (rc); -} - /*************************************************************************** - Function Name: _MPIX_statistics_write - Description: Call MPIDI_Statistics_write to write statistical - information to specified file descriptor. - Parameters: - Name Type I/O - fptr FILE* I File pointer, can be stdout or stderr. - If it is to a file, user has to open - the file. - rc (Fortran only) int 0 Return sum from MPIDI_Statistics_write calls - <returns> (C only) 0 Both MPICH and PAMI statistics - ***************************************************************************/ -int _MPIX_statistics_write(FILE* fptr) -{ - int rc = MPIDI_Statistics_write(fptr); - if (rc < 0) { - MPID_assert(rc == PAMI_SUCCESS); - } - return(rc); -} - -int MPIX_statistics_write(FILE* fptr) -{ - return(_MPIX_statistics_write(fptr)); -} - -/* Fortran: fdes is pointer to a file descriptor. - * rc is pointer to buffer for storing return code. - * - * Note: Fortran app. will convert a Fortran I/O unit to a file - * descriptor by calling Fortran utilities, flush_ and getfd. - * When fdes=1, output is to STDOUT. When fdes=2, output is to STDERR. - */ - -void MPIXF_statistics_write(int *fdes, int *rc) -{ - FILE *fp; - int dup_fd; - int closefp=0; - - /* Convert the DUP file descriptor to a FILE pointer */ - dup_fd = dup(*fdes); - if ( (fp = fdopen(dup_fd, "a")) != NULL ) - closefp = 1; - else - fp = stdout; /* If fdopen failed then default to stdout */ - - *rc = _MPIX_statistics_write(fp); - - /* The check is because I don't want to close stdout. */ - if ( closefp ) fclose(fp); -} - -void MPIXF_statistics_write_(int *fdes, int *rc) -{ - FILE *fp; - int dup_fd; - int closefp=0; - - /* Convert the DUP file descriptor to a FILE pointer */ - dup_fd = dup(*fdes); - if ( (fp = fdopen(dup_fd, "a")) != NULL ) - closefp = 1; - else - fp = stdout; /* If fdopen failed then default to stdout */ - - *rc = _MPIX_statistics_write(fp); - - /* The check is because I don't want to close stdout. */ - if ( closefp ) fclose(fp); -} -#endif - - -#ifdef __BGQ__ - -int -MPIX_Torus_ndims(int *numdimensions) -{ - const pamix_torus_info_t *info = PAMIX_Torus_info(); - *numdimensions = info->dims - 1; - return MPI_SUCCESS; -} - - -int -MPIX_Rank2torus(int rank, int *coords) -{ - size_t coord_array[MPIDI_HW.torus_dimension+1]; - - PAMIX_Task2torus(rank, coord_array); - - unsigned i; - for(i=0; i<MPIDI_HW.torus_dimension+1; ++i) - coords[i] = coord_array[i]; - - return MPI_SUCCESS; -} - - -int -MPIX_Torus2rank(int *coords, int *rank) -{ - size_t coord_array[MPIDI_HW.torus_dimension+1]; - pami_task_t task; - - unsigned i; - for(i=0; i<MPIDI_HW.torus_dimension+1; ++i) - coord_array[i] = coords[i]; - - PAMIX_Torus2task(coord_array, &task); - *rank = task; - - return MPI_SUCCESS; -} - - -typedef struct -{ - pami_geometry_t geometry; - pami_work_t state; - pami_configuration_t config; - size_t num_configs; - pami_event_function fn; - void* cookie; -} MPIX_Comm_update_data_t; - -static void -MPIX_Comm_update_done(void *ctxt, void *data, pami_result_t err) -{ - int *active = (int *)data; - (*active)--; -} - -static pami_result_t -MPIX_Comm_update_post(pami_context_t context, void *cookie) -{ - MPIX_Comm_update_data_t *data = (MPIX_Comm_update_data_t *)cookie; - - return PAMI_Geometry_update(data->geometry, - &data->config, - data->num_configs, - context, - data->fn, - data->cookie); -} - -int -MPIX_Comm_update(MPI_Comm comm, int optimize) -{ - MPIR_Comm * comm_ptr; - volatile int geom_update = 1; - MPIX_Comm_update_data_t data; - pami_configuration_t config; - - MPIR_Comm_get_ptr(comm, comm_ptr); - if (!comm_ptr || comm == MPI_COMM_NULL) - return MPI_ERR_COMM; - - /* First, check if there is a geometry. When optimized collectives - * are disabled, no geometry is created */ - if(comm_ptr->mpid.geometry == NULL) - return MPI_ERR_COMM; - - config.name = PAMI_GEOMETRY_OPTIMIZE; - config.value.intval = !!optimize; - - TRACE_ERR("About to %s geometry update function\n", MPIDI_Process.context_post.active>0?"post":"invoke"); - data.num_configs = 1; - data.config.name = config.name; - data.config.value.intval = config.value.intval; - data.fn = MPIX_Comm_update_done; - data.cookie = (void *)&geom_update; - data.geometry = comm_ptr->mpid.geometry; - MPIDI_Context_post(MPIDI_Context[0], - &data.state, - MPIX_Comm_update_post, - &data); - TRACE_ERR("Geometry update function %s\n", MPIDI_Process.context_post.active>0?"posted":"invoked"); - - TRACE_ERR("Waiting for geometry update to finish\n"); - - MPID_PROGRESS_WAIT_WHILE(geom_update); - - MPIDI_Comm_coll_query(comm_ptr); - MPIDI_Comm_coll_envvars(comm_ptr); - if(MPIDI_Process.optimized.select_colls) - MPIDI_Comm_coll_select(comm_ptr); - - return MPI_SUCCESS; -} - -int -MPIX_Get_last_algorithm_name(MPI_Comm comm, char *protocol, int length) -{ - MPIR_Comm *comm_ptr; - MPIR_Comm_get_ptr(comm, comm_ptr); - - if(!comm_ptr || comm == MPI_COMM_NULL) - return MPI_ERR_COMM; - if(!protocol || length <= 0) - return MPI_ERR_ARG; - strncpy(protocol, comm_ptr->mpid.last_algorithm, length); - return MPI_SUCCESS; -} - -#undef FUNCNAME -#define FUNCNAME MPIX_IO_node_id -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIX_IO_node_id () -{ - static unsigned long IO_node_id = ULONG_MAX; - - if (IO_node_id != ULONG_MAX) - return (int)(IO_node_id>>32); - - int rc; - int fd; - char* uci_str; - char buffer[4096]; - - fd = open("/dev/bgpers", O_RDONLY, 0); - assert(fd>=0); - rc = read(fd, buffer, sizeof(buffer)); - assert(rc>0); - close(fd); - - uci_str = strstr(buffer, "BG_UCI="); - assert(uci_str); - uci_str += sizeof("BG_UCI=")-1; - - IO_node_id = strtoul(uci_str, NULL, 16); - return (int)(IO_node_id>>32); -} - -#undef FUNCNAME -#define FUNCNAME MPIX_IO_link_id -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIX_IO_link_id () -{ - int nB, nC, nD, nE; /* Size of each torus dimension */ - int brA, brB, brC, brD, brE; /* The bridge node's coordinates */ - - Personality_t personality; - - Kernel_GetPersonality(&personality, sizeof(personality)); - - nB = personality.Network_Config.Bnodes; - nC = personality.Network_Config.Cnodes; - nD = personality.Network_Config.Dnodes; - nE = personality.Network_Config.Enodes; - - brA = personality.Network_Config.cnBridge_A; - brB = personality.Network_Config.cnBridge_B; - brC = personality.Network_Config.cnBridge_C; - brD = personality.Network_Config.cnBridge_D; - brE = personality.Network_Config.cnBridge_E; - - /* - * This is the bridge node, numbered in ABCDE order, E increments first. - * It is considered the unique "io node route identifer" because each - * bridge node only has one torus link to one io node. - */ - return brE + brD*nE + brC*nD*nE + brB*nC*nD*nE + brA*nB*nC*nD*nE; -}; - -#undef FUNCNAME -#define FUNCNAME MPIX_IO_distance -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPIX_IO_distance () -{ - int iA, iB, iC, iD, iE; /* The local node's coordinates */ - int nA, nB, nC, nD, nE; /* Size of each torus dimension */ - int brA, brB, brC, brD, brE; /* The bridge node's coordinates */ - int Nflags; - int d1, d2; - int dA, dB, dC, dD, dE; /* distance from local node to bridge node */ - - Personality_t personality; - - Kernel_GetPersonality(&personality, sizeof(personality)); - - iA = personality.Network_Config.Acoord; - iB = personality.Network_Config.Bcoord; - iC = personality.Network_Config.Ccoord; - iD = personality.Network_Config.Dcoord; - iE = personality.Network_Config.Ecoord; - - nA = personality.Network_Config.Anodes; - nB = personality.Network_Config.Bnodes; - nC = personality.Network_Config.Cnodes; - nD = personality.Network_Config.Dnodes; - nE = personality.Network_Config.Enodes; - - brA = personality.Network_Config.cnBridge_A; - brB = personality.Network_Config.cnBridge_B; - brC = personality.Network_Config.cnBridge_C; - brD = personality.Network_Config.cnBridge_D; - brE = personality.Network_Config.cnBridge_E; - - Nflags = personality.Network_Config.NetFlags; - - dA = abs(iA - brA); - if (Nflags & ND_ENABLE_TORUS_DIM_A) - { - d1 = dA; - d2 = nA - d1; - dA = (d1 < d2) ? d1 : d2; - } - - dB = abs(iB - brB); - if (Nflags & ND_ENABLE_TORUS_DIM_B) - { - d1 = dB; - d2 = nB - d1; - dB = (d1 < d2) ? d1 : d2; - } - - dC = abs(iC - brC); - if (Nflags & ND_ENABLE_TORUS_DIM_C) - { - d1 = dC; - d2 = nC - d1; - dC = (d1 < d2) ? d1 : d2; - } - - dD = abs(iD - brD); - if (Nflags & ND_ENABLE_TORUS_DIM_D) - { - d1 = dD; - d2 = nD - d1; - dD = (d1 < d2) ? d1 : d2; - } - - dE = abs(iE - brE); - if (Nflags & ND_ENABLE_TORUS_DIM_E) - { - d1 = dE; - d2 = nE - d1; - dE = (d1 < d2) ? d1 : d2; - } - - /* This is the number of hops to the io node */ - return dA + dB + dC + dD + dE + 1; -}; - -#undef FUNCNAME -#define FUNCNAME MPIX_Pset_io_node -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -void -MPIX_Pset_io_node (int *io_node_route_id, int *distance_to_io_node) -{ - *io_node_route_id = MPIX_IO_link_id (); - *distance_to_io_node = MPIX_IO_distance (); - - return; -}; - -/** - * \brief Create a communicator of ranks that have a common bridge node. - * - * \note This function is private to this source file. - * - * \param [in] parent_comm_ptr Pointer to the parent communicator - * \param [out] pset_comm_ptr Pointer to the new 'MPID' communicator - * - * \return MPI status - */ -int _MPIX_Pset_same_comm_create (MPIR_Comm *parent_comm_ptr, MPIR_Comm **pset_comm_ptr) -{ - int color, key; - int mpi_errno; - - color = MPIX_IO_link_id (); - key = MPIX_IO_distance (); - - /* - * Use MPIR_Comm_split_impl to make a communicator of all ranks in the parent - * communicator that share the same bridge node; i.e. the 'color' is the - * 'io node route identifer', which is unique to each BGQ bridge node. - * - * Setting the 'key' to the 'distance to io node' ensures that rank 0 in - * the new communicator is on the bridge node, or as close to the bridge node - * as possible. - */ - - *pset_comm_ptr = NULL; - mpi_errno = MPI_SUCCESS; - mpi_errno = MPIR_Comm_split_impl(parent_comm_ptr, color, key, pset_comm_ptr); - - return mpi_errno; -} - -#undef FUNCNAME -#define FUNCNAME MPIX_Pset_same_comm_create_from_parent -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPIX_Pset_same_comm_create_from_parent (MPI_Comm parent_comm, MPI_Comm *pset_comm) -{ - int mpi_errno; - MPIR_Comm *parent_comm_ptr, *pset_comm_ptr; - - *pset_comm = MPI_COMM_NULL; - - /* - * Convert the parent communicator object handle to an object pointer; - * needed by the error handling code. - */ - parent_comm_ptr = NULL; - MPIR_Comm_get_ptr(parent_comm, parent_comm_ptr); - - mpi_errno = MPI_SUCCESS; - mpi_errno = _MPIX_Pset_same_comm_create (parent_comm_ptr, &pset_comm_ptr); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - if (pset_comm_ptr) - MPIR_OBJ_PUBLISH_HANDLE(*pset_comm, pset_comm_ptr->handle); - else - goto fn_fail; - -fn_exit: - return mpi_errno; -fn_fail: - mpi_errno = MPIR_Err_return_comm( parent_comm_ptr, FCNAME, mpi_errno ); - goto fn_exit; -}; - -#undef FUNCNAME -#define FUNCNAME MPIX_Pset_same_comm_create -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPIX_Pset_same_comm_create (MPI_Comm *pset_comm) -{ - return MPIX_Pset_same_comm_create_from_parent (MPI_COMM_WORLD, pset_comm); -}; - - -#undef FUNCNAME -#define FUNCNAME MPIX_Pset_diff_comm_create_from_parent -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPIX_Pset_diff_comm_create_from_parent (MPI_Comm parent_comm, MPI_Comm *pset_comm) -{ - MPIR_Comm *parent_comm_ptr, *pset_same_comm_ptr, *pset_diff_comm_ptr; - int color, key; - int mpi_errno; - - *pset_comm = MPI_COMM_NULL; - - /* - * Convert the parent communicator object handle to an object pointer; - * needed by the error handling code. - */ - parent_comm_ptr = NULL; - MPIR_Comm_get_ptr(parent_comm, parent_comm_ptr); - - /* - * Determine the 'color' of this rank to create the new communicator - which - * is the rank in a (transient) communicator where all ranks share a common - * bridge node. - */ - mpi_errno = MPI_SUCCESS; - mpi_errno = _MPIX_Pset_same_comm_create (parent_comm_ptr, &pset_same_comm_ptr); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - if (pset_same_comm_ptr == NULL) - goto fn_fail; - - color = MPIR_Comm_rank(pset_same_comm_ptr) * MPIDI_HW.ppn + MPIDI_HW.coreID; - - /* Discard the 'pset_same_comm_ptr' .. it is no longer needed. */ - mpi_errno = MPIR_Comm_free_impl(pset_same_comm_ptr); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - /* Set the 'key' for this rank to order the ranks in the new communicator. */ - key = MPIR_Comm_rank(parent_comm_ptr); - - pset_diff_comm_ptr = NULL; - mpi_errno = MPI_SUCCESS; - mpi_errno = MPIR_Comm_split_impl(parent_comm_ptr, color, key, &pset_diff_comm_ptr); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - if (pset_diff_comm_ptr) - MPIR_OBJ_PUBLISH_HANDLE(*pset_comm, pset_diff_comm_ptr->handle); - else - goto fn_fail; - -fn_exit: - return mpi_errno; -fn_fail: - mpi_errno = MPIR_Err_return_comm( parent_comm_ptr, FCNAME, mpi_errno ); - goto fn_exit; -}; - -#undef FUNCNAME -#define FUNCNAME MPIX_Pset_diff_comm_create -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPIX_Pset_diff_comm_create (MPI_Comm *pset_comm) -{ - return MPIX_Pset_diff_comm_create_from_parent (MPI_COMM_WORLD, pset_comm); -}; - - - -/** - * \brief Compare each elemt of two six-element arrays - * \param [in] A The first array - * \param [in] B The first array - * \return MPI_SUCCESS (does not return on failure) - */ -#define CMP_6(A,B) \ -({ \ - assert(A[0] == B[0]); \ - assert(A[1] == B[1]); \ - assert(A[2] == B[2]); \ - assert(A[3] == B[3]); \ - assert(A[4] == B[4]); \ - assert(A[5] == B[5]); \ - MPI_SUCCESS; \ -}) - -#undef FUNCNAME -#define FUNCNAME MPIX_Cart_comm_create -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPIX_Cart_comm_create (MPI_Comm *cart_comm) -{ - int result; - int rank, numprocs, - dims[6], - wrap[6], - coords[6]; - int new_rank1, new_rank2; - MPI_Comm new_comm = MPI_COMM_NULL; - int cart_rank, - cart_dims[6], - cart_wrap[6], - cart_coords[6]; - int Nflags; - - *cart_comm = MPI_COMM_NULL; - PMPI_Comm_rank(MPI_COMM_WORLD, &rank); - PMPI_Comm_size(MPI_COMM_WORLD, &numprocs); - - Personality_t personality; - - Kernel_GetPersonality(&personality, sizeof(personality)); - - dims[0] = personality.Network_Config.Anodes; - dims[1] = personality.Network_Config.Bnodes; - dims[2] = personality.Network_Config.Cnodes; - dims[3] = personality.Network_Config.Dnodes; - dims[4] = personality.Network_Config.Enodes; - dims[5] = Kernel_ProcessCount(); - - /* This only works if MPI_COMM_WORLD is the full partition */ - if (dims[5] * dims[4] * dims[3] * dims[2] * dims[1] * dims[0] != numprocs) - return MPI_ERR_TOPOLOGY; - - Nflags = personality.Network_Config.NetFlags; - wrap[0] = ((Nflags & ND_ENABLE_TORUS_DIM_A) != 0); - wrap[1] = ((Nflags & ND_ENABLE_TORUS_DIM_B) != 0); - wrap[2] = ((Nflags & ND_ENABLE_TORUS_DIM_C) != 0); - wrap[3] = ((Nflags & ND_ENABLE_TORUS_DIM_D) != 0); - wrap[4] = ((Nflags & ND_ENABLE_TORUS_DIM_E) != 0); - wrap[5] = 1; - - coords[0] = personality.Network_Config.Acoord; - coords[1] = personality.Network_Config.Bcoord; - coords[2] = personality.Network_Config.Ccoord; - coords[3] = personality.Network_Config.Dcoord; - coords[4] = personality.Network_Config.Ecoord; - coords[5] = Kernel_MyTcoord(); - - new_rank1 = coords[5] + - dims[5] * coords[4] + - dims[5] * dims[4] * coords[3] + - dims[5] * dims[4] * dims[3] * coords[2] + - dims[5] * dims[4] * dims[3] * dims[2] * coords[1] + - dims[5] * dims[4] * dims[3] * dims[2] * dims[1] * coords[0]; - - result = PMPI_Comm_split(MPI_COMM_WORLD, 0, new_rank1, &new_comm); - if (result != MPI_SUCCESS) - { - PMPI_Comm_free(&new_comm); - return result; - } - PMPI_Comm_rank(new_comm, &new_rank2); - assert(new_rank1 == new_rank2); - - result = PMPI_Cart_create(new_comm, - 6, - dims, - wrap, - 0, - cart_comm); - if (result != MPI_SUCCESS) - return result; - - PMPI_Comm_rank(*cart_comm, &cart_rank); - PMPI_Cart_get (*cart_comm, 6, cart_dims, cart_wrap, cart_coords); - - CMP_6(dims, cart_dims); - CMP_6(wrap, cart_wrap); - CMP_6(coords, cart_coords); - - PMPI_Comm_free(&new_comm); - return MPI_SUCCESS; -}; - -/** - * \brief FORTRAN interface to MPIX_Comm_rank2global - * - * \param [in] comm Communicator - * \param [in] crank Pointer to the rank in the communicator variable - * \param [out] grank Pointer tot he global rank variable - * - * \return status - */ -int mpix_comm_rank2global (MPI_Comm *comm, int *crank, int *grank) -{ - return MPIX_Comm_rank2global (*comm, *crank, grank); -} - -/** - * \brief FORTRAN interface to MPIX_Pset_same_comm_create - * - * \param [out] pset_comm Communicator - * - * \return status - */ -int mpix_pset_same_comm_create (MPI_Comm *pset_comm) -{ - return MPIX_Pset_same_comm_create (pset_comm); -} - -/** - * \brief FORTRAN interface to MPIX_Pset_diff_comm_create - * - * \param [out] pset_comm Communicator - * - * \return status - */ -int mpix_pset_diff_comm_create (MPI_Comm *pset_comm) -{ - return MPIX_Pset_diff_comm_create (pset_comm); -} - -/** - * \brief FORTRAN interface to MPIX_Pset_same_comm_create_from_parent - * - * \param [in] parent_comm Parent communicator - * \param [out] pset_comm New pset communicator - * - * \return status - */ -int mpix_pset_same_comm_create_from_parent (MPI_Comm *parent_comm, MPI_Comm *pset_comm) -{ - return MPIX_Pset_same_comm_create_from_parent (*parent_comm, pset_comm); -} - -/** - * \brief FORTRAN interface to MPIX_Pset_diff_comm_create_from_parent - * - * \param [in] parent_comm Parent communicator - * \param [out] pset_comm New pset communicator - * - * \return status - */ -int mpix_pset_diff_comm_create_from_parent (MPI_Comm *parent_comm, MPI_Comm *pset_comm) -{ - return MPIX_Pset_diff_comm_create_from_parent (*parent_comm, pset_comm); -} - -/** - * \brief FORTRAN interface to MPIX_IO_node_id - * - * \param [out] io_node_id This rank's io node id - */ -void mpix_io_node_id (int *io_node_id) { *io_node_id = MPIX_IO_node_id(); } - -/** - * \brief FORTRAN interface to MPIX_IO_link_id - * - * \param [out] io_link_id This rank's io link id - */ -void mpix_io_link_id (int *io_link_id) { *io_link_id = MPIX_IO_link_id(); } - -/** - * \brief FORTRAN interface to MPIX_IO_distance - * - * \param [out] io_distance This rank's distance to the io node - */ -void mpix_io_distance (int *io_distance) { *io_distance = MPIX_IO_distance(); } - -/** - * \brief FORTRAN interface to MPIX_Cart_comm_create - * - * \param [out] cart_comm Communicator to create - * - * \return status - */ -int mpix_cart_comm_create (MPI_Comm *cart_comm) -{ - return MPIX_Cart_comm_create (cart_comm); -} - - - -#endif - -#ifdef __PE__ -void mpc_disableintr() __attribute__ ((alias("MPIX_disableintr"))); -void mp_disableintr() __attribute__ ((alias("MPIXF_disableintr"))); -void mp_disableintr_() __attribute__ ((alias("MPIXF_disableintr"))); -void mp_disableintr__() __attribute__ ((alias("MPIXF_disableintr"))); -void mpc_enableintr() __attribute__ ((alias("MPIX_enableintr"))); -void mp_enableintr() __attribute__ ((alias("MPIXF_enableintr"))); -void mp_enableintr_() __attribute__ ((alias("MPIXF_enableintr"))); -void mp_enableintr__() __attribute__ ((alias("MPIXF_enableintr"))); -void mpc_queryintr() __attribute__ ((weak,alias("MPIX_queryintr"))); -void mp_queryintr() __attribute__ ((alias("MPIXF_queryintr"))); -void mp_queryintr_() __attribute__ ((alias("MPIXF_queryintr"))); -void mp_queryintr__() __attribute__ ((alias("MPIXF_queryintr"))); - - /*************************************************************************** - Function Name: MPIX_disableintr - - Description: Call the pamid layer to disable interrupts. - (Similar to setting MP_CSS_INTERRUPT to "no") - - Parameters: The Fortran versions have an int* parameter used to pass the - return code to the calling program. - - Returns: 0 Success - <0 Failure - ***************************************************************************/ - -int -_MPIDI_disableintr() -{ - return(MPIDI_disableintr()); -} - -int -MPIX_disableintr() -{ - return(_MPIDI_disableintr()); -} - -void -MPIXF_disableintr(int *rc) -{ - *rc = _MPIDI_disableintr(); -} - -void -MPIXF_disableintr_(int *rc) -{ - *rc = _MPIDI_disableintr(); -} - -/* - ** Called by: _mp_disableintr - ** Purpose : Disables interrupts - */ -int -MPIDI_disableintr() -{ - pami_result_t rc=0; - int i; - - MPIR_ERRTEST_INITIALIZED_ORDIE(); - if (MPIDI_Process.mp_interrupts!= 0) - { - TRACE_ERR("Async advance beginning...\n"); - /* Enable async progress on all contexts.*/ - for (i=0; i<MPIDI_Process.avail_contexts; ++i) - { - PAMIX_Progress_disable(MPIDI_Context[i], PAMIX_PROGRESS_ALL); - } - TRACE_ERR("Async advance disabled\n"); - MPIDI_Process.mp_interrupts=0; - } - return(rc); -} - /*************************************************************************** - Function Name: MPIX_enableintr - - Description: Call the pamid-layer function to enable interrupts. - (Similar to setting MP_CSS_INTERRUPT to "yes") - - Parameters: The Fortran versions have an int* parameter used to pass the - return code to the calling program. - - Returns: 0 Success - <0 Failure - ***************************************************************************/ -int -_MPIDI_enableintr() -{ - return(MPIDI_enableintr()); -} - -/* C callable version */ -int -MPIX_enableintr() -{ - return(_MPIDI_enableintr()); -} - -/* Fortran callable version */ -void -MPIXF_enableintr(int *rc) -{ - *rc = _MPIDI_enableintr(); -} - -/* Fortran callable version for -qEXTNAME support */ -void -MPIXF_enableintr_(int *rc) -{ - *rc = _MPIDI_enableintr(); -} - -int -MPIDI_enableintr() -{ - pami_result_t rc=0; - int i; - - MPIR_ERRTEST_INITIALIZED_ORDIE(); - if (MPIDI_Process.mp_interrupts == 0) - { - /* Enable async progress on all contexts.*/ - for (i=0; i<MPIDI_Process.avail_contexts; ++i) - { - PAMIX_Progress_enable(MPIDI_Context[i], PAMIX_PROGRESS_ALL); - } - TRACE_ERR("Async advance enabled\n"); - MPIDI_Process.mp_interrupts=1; - } - MPID_assert(rc == PAMI_SUCCESS); - return(rc); -} - - /*************************************************************************** - Function Name: MPIX_queryintr - - Description: Call the pamid-layer function to determine if - interrupts are currently on or off. - - Parameters: The Fortran versions have an int* parameter used to pass the - current interrupt setting to the calling program. - Returns: 0 Indicates interrupts are currently off - 1 Indicates interrupts are currently on - <0 Failure - ***************************************************************************/ -int -MPIDI_queryintr() -{ - return(MPIDI_Process.mp_interrupts); -} - -int -_MPIDI_queryintr() -{ - return(MPIDI_queryintr()); -} - -int -MPIX_queryintr() -{ - return(_MPIDI_queryintr()); -} - -void -MPIXF_queryintr(int *rc) -{ - *rc = _MPIDI_queryintr(); -} - -void -MPIXF_queryintr_(int *rc) -{ - *rc = _MPIDI_queryintr(); -} -#endif diff --git a/src/mpid/pamid/src/onesided/Makefile.mk b/src/mpid/pamid/src/onesided/Makefile.mk deleted file mode 100644 index c58ca32..0000000 --- a/src/mpid/pamid/src/onesided/Makefile.mk +++ /dev/null @@ -1,57 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -noinst_HEADERS += \ - src/mpid/pamid/src/onesided/mpidi_onesided.h - - -mpi_core_sources += \ - src/mpid/pamid/src/onesided/mpid_1s.c \ - src/mpid/pamid/src/onesided/mpid_win_accumulate.c \ - src/mpid/pamid/src/onesided/mpid_win_create.c \ - src/mpid/pamid/src/onesided/mpid_win_fence.c \ - src/mpid/pamid/src/onesided/mpid_win_free.c \ - src/mpid/pamid/src/onesided/mpid_win_get.c \ - src/mpid/pamid/src/onesided/mpid_win_lock.c \ - src/mpid/pamid/src/onesided/mpid_win_lock_all.c \ - src/mpid/pamid/src/onesided/mpid_win_pscw.c \ - src/mpid/pamid/src/onesided/mpid_win_put.c \ - src/mpid/pamid/src/onesided/mpid_win_shared_query.c \ - src/mpid/pamid/src/onesided/mpid_win_create_dynamic.c \ - src/mpid/pamid/src/onesided/mpid_win_allocate_shared.c \ - src/mpid/pamid/src/onesided/mpid_win_flush.c \ - src/mpid/pamid/src/onesided/mpid_win_allocate.c \ - src/mpid/pamid/src/onesided/mpid_win_sync.c \ - src/mpid/pamid/src/onesided/mpid_win_attach.c \ - src/mpid/pamid/src/onesided/mpid_win_detach.c \ - src/mpid/pamid/src/onesided/mpid_win_get_info.c \ - src/mpid/pamid/src/onesided/mpid_win_set_info.c \ - src/mpid/pamid/src/onesided/mpid_win_get_accumulate.c \ - src/mpid/pamid/src/onesided/mpid_win_reqops.c \ - src/mpid/pamid/src/onesided/mpidi_win_control.c \ - src/mpid/pamid/src/onesided/mpid_win_compare_and_swap.c \ - src/mpid/pamid/src/onesided/mpid_win_fetch_and_op.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/onesided/mpid_1s.c b/src/mpid/pamid/src/onesided/mpid_1s.c deleted file mode 100644 index e9a49b0..0000000 --- a/src/mpid/pamid/src/onesided/mpid_1s.c +++ /dev/null @@ -1,151 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_1s.c - * \brief ??? - */ -#include "mpidi_onesided.h" - - -void -MPIDI_Win_DoneCB(pami_context_t context, - void * cookie, - pami_result_t result) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)cookie; - ++req->win->mpid.sync.complete; - ++req->origin.completed; - - if ((req->buffer_free) && (req->type == MPIDI_WIN_REQUEST_GET)) - { - if (req->origin.completed == req->target.dt.num_contig) - { - int mpi_errno; - mpi_errno = MPIR_Localcopy(req->buffer, - req->origin.dt.size, - MPI_CHAR, - req->origin.addr, - req->origin.count, - req->origin.datatype); - MPID_assert(mpi_errno == MPI_SUCCESS); - MPIDU_Datatype_release(req->origin.dt.pointer); - MPL_free(req->buffer); - MPL_free(req->user_buffer); - req->buffer_free = 0; - } - } - - - if ((req->origin.completed == req->target.dt.num_contig) || - ((req->type >= MPIDI_WIN_REQUEST_COMPARE_AND_SWAP) && - (req->origin.completed == req->origin.dt.num_contig))) - { - MPIR_Request * req_handle = req->req_handle; - - if (req->buffer_free) { - MPL_free(req->buffer); - MPL_free(req->user_buffer); - req->buffer_free = 0; - } - MPIDI_Win_datatype_unmap(&req->target.dt); - if (req->accum_headers) - MPL_free(req->accum_headers); - - if (!((req->type > MPIDI_WIN_REQUEST_GET_ACCUMULATE) && (req->type <=MPIDI_WIN_REQUEST_RGET_ACCUMULATE))) - MPL_free(req); - - if(req_handle) { - - /* The instant this completion counter is set to zero another thread - * may notice the change and begin freeing request resources. The - * thread executing the code in this function must not touch any - * portion of the request structure after decrementing the completion - * counter. - * - * See MPID_Request_free_inline() - */ - MPIR_cc_set(req_handle->cc_ptr, 0); - } - } - if ((MPIDI_Process.typed_onesided == 1) && (!req->target.dt.contig || !req->origin.dt.contig)) { - /* We used the PAMI_Rput/Rget_typed call and added a ref so any MPI_Type_free before the context - * executes the put/get would not free the MPIDU_Datatype, which would also free the associated PAMI datatype - * which was still needed for communication -- that communication has completed, so now release the ref - * in the callback to allow the MPIDU_Datatypeto be freed. - */ - MPIDU_Datatype_release(req->origin.dt.pointer); - MPIDU_Datatype_release(req->target.dt.pointer); - } - MPIDI_Progress_signal(); -} - - -void -MPIDI_Win_datatype_basic(int count, - MPI_Datatype datatype, - MPIDI_Datatype * dt) -{ - MPIDI_Datatype_get_info(dt->count = count, - dt->type = datatype, - dt->contig, - dt->size, - dt->pointer, - dt->true_lb); - TRACE_ERR("DT=%08x DTp=%p count=%d contig=%d size=%zu true_lb=%zu\n", - dt->type, dt->pointer, dt->count, dt->contig, (size_t)dt->size, (size_t)dt->true_lb); -} - - -void -MPIDI_Win_datatype_map(MPIDI_Datatype * dt) -{ - if (dt->contig) - { - dt->num_contig = 1; - dt->map = &dt->__map; - dt->map[0].DLOOP_VECTOR_BUF = (void*)(size_t)dt->true_lb; - dt->map[0].DLOOP_VECTOR_LEN = dt->size; - } - else - { - unsigned map_size = dt->pointer->max_contig_blocks*dt->count + 1; - dt->num_contig = map_size; - dt->map = (DLOOP_VECTOR*)MPL_malloc(map_size * sizeof(DLOOP_VECTOR)); - MPID_assert(dt->map != NULL); - - DLOOP_Offset last = dt->pointer->size*dt->count; - MPIDU_Segment seg; - MPIDU_Segment_init(NULL, dt->count, dt->type, &seg, 0); - MPIDU_Segment_pack_vector(&seg, 0, &last, dt->map, &dt->num_contig); - MPID_assert((unsigned)dt->num_contig <= map_size); -#ifdef TRACE_ON - TRACE_ERR("dt->pointer->size=%d num_contig: orig=%u new=%d\n", dt->pointer->size, map_size, dt->num_contig); - int i; - for(i=0; i<dt->num_contig; ++i) - TRACE_ERR(" %d: BUF=%zu LEN=%zu\n", i, (size_t)dt->map[i].DLOOP_VECTOR_BUF, (size_t)dt->map[i].DLOOP_VECTOR_LEN); -#endif - } -} - - -void -MPIDI_Win_datatype_unmap(MPIDI_Datatype * dt) -{ - if (dt->map != &dt->__map) - MPL_free(dt->map);; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_accumulate.c b/src/mpid/pamid/src/onesided/mpid_win_accumulate.c deleted file mode 100644 index c833e60..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_accumulate.c +++ /dev/null @@ -1,364 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_accumulate.c - * \brief ??? - */ -#include "mpidi_onesided.h" -#include "mpidi_util.h" - - -void -MPIDI_WinAccumCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv != NULL); - MPID_assert(sndbuf == NULL); - MPID_assert(msginfo_size == sizeof(MPIDI_Win_MsgInfo)); - MPID_assert(_msginfo != NULL); - const MPIDI_Win_MsgInfo * msginfo = (const MPIDI_Win_MsgInfo*)_msginfo; - - int null=0; - pami_type_t pami_type; - pami_data_function pami_op; - MPIDI_Datatype_to_pami(msginfo->type, &pami_type, msginfo->op, &pami_op, &null); - -#ifdef TRACE_ON - void * buf = msginfo->addr; - unsigned* ibuf = (unsigned*)buf; - double * dbuf = (double *)buf; - TRACE_ERR("New accum msg: len=%zu type=%x op=%x l-buf=%p *(int*)buf=0x%08x *(double*)buf=%g\n", sndlen, msginfo->type, msginfo->op, buf, *ibuf, *dbuf); - TRACE_ERR(" PAMI: type=%p op=%p\n", pami_type, pami_op); -#endif - - MPID_assert(recv != NULL); - *recv = zero_recv_parms; - recv->cookie = NULL; - recv->local_fn = NULL; - recv->addr = msginfo->addr; - recv->type = pami_type; - recv->offset = 0; - recv->data_fn = pami_op; - recv->data_cookie = NULL; -} - - -static pami_result_t -MPIDI_Accumulate(pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - void *map; - pami_send_t params; - - params = zero_send_parms; - params.send.header.iov_len = sizeof(MPIDI_Win_MsgInfo); - params.send.dispatch = MPIDI_Protocols_WinAccum; - params.send.dest = req->dest; - params.events.cookie = req; - params.events.remote_fn = MPIDI_Win_DoneCB; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - TRACE_ERR("Start index=%u/%d l-addr=%p r-base=%p r-offset=%zu (sync->started=%u sync->complete=%u)\n", - req->state.index, req->target.dt.num_contig, req->buffer, req->win->mpid.info[req->target.rank].base_addr, req->offset, sync->started, sync->complete); - while (req->state.index < req->target.dt.num_contig) { - if (sync->started > sync->complete + MPIDI_Process.rma_pending) - { - TRACE_ERR("Bailing out; index=%u/%d sync->started=%u sync->complete=%u\n", - req->state.index, req->target.dt.num_contig, sync->started, sync->complete); - return PAMI_EAGAIN; - } - ++sync->started; - - - params.send.header.iov_base = &(((MPIDI_Win_MsgInfo *)req->accum_headers)[req->state.index]); - params.send.data.iov_len = req->target.dt.map[req->state.index].DLOOP_VECTOR_LEN; - params.send.data.iov_base = req->buffer + req->state.local_offset; - -#ifdef TRACE_ON - void * buf = params.send.data.iov_base; - unsigned* ibuf = (unsigned*)buf; - double * dbuf = (double *)buf; - TRACE_ERR(" Sub index=%u bytes=%zu l-offset=%zu r-addr=%p l-buf=%p *(int*)buf=0x%08x *(double*)buf=%g\n", - req->state.index, params.send.data.iov_len, req->state.local_offset, req->accum_headers[req->state.index].addr, buf, *ibuf, *dbuf); -#endif - /** sync->total will be updated with every RMA and the complete - will not change till that RMA has completed. In the meanwhile - the rest of the RMAs will have memory leaks */ - if (req->target.dt.num_contig - req->state.index == 1) { - rc = PAMI_Send(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; - } else { - rc = PAMI_Send(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - req->state.local_offset += params.send.data.iov_len; - ++req->state.index; - } - } - - - return PAMI_SUCCESS; -} - - -/** - * \brief MPI-PAMI glue for MPI_ACCUMULATE function - * - * According to the MPI Specification: - * - * Each datatype argument must be a predefined datatype or - * a derived datatype, where all basic components are of the - * same predefined datatype. Both datatype arguments must be - * constructed from the same predefined datatype. - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] op Operand to perform - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Accumulate -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Accumulate(const void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPI_Op op, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - int shm_locked = 0; - MPIDI_Win_request *req = MPL_calloc0(1, MPIDI_Win_request); - req->win = win; - if(win->mpid.request_based != 1) - req->type = MPIDI_WIN_REQUEST_ACCUMULATE; - else { - req->type = MPIDI_WIN_REQUEST_RACCUMULATE; - req->req_handle = win->mpid.rreq; - req->req_handle->mpid.win_req = req; - } - - if(win->mpid.sync.origin_epoch_type == win->mpid.sync.target_epoch_type && - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_FENCE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_FENCE; - } - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_NONE || - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - req->offset = target_disp * win->mpid.info[target_rank].disp_unit; -#ifdef __BGQ__ - /* PAMI limitation as it doesnt permit VA of 0 to be passed into - * memregion create, so we must pass base_va of heap computed from - * an SPI call instead. So the target offset must be adjusted */ - if (req->win->create_flavor == MPI_WIN_FLAVOR_DYNAMIC) - req->offset -= (size_t)req->win->mpid.info[target_rank].base_addr; -#endif - - if (origin_datatype == MPI_DOUBLE_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_DOUBLE, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_DOUBLE, - &req->target.dt); - } - else if (origin_datatype == MPI_LONG_DOUBLE_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_LONG_DOUBLE, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_LONG_DOUBLE, - &req->target.dt); - } - else if (origin_datatype == MPI_LONG_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_LONG, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_LONG, - &req->target.dt); - } - else if (origin_datatype == MPI_SHORT_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_INT, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_INT, - &req->target.dt); - } - else - { - MPIDI_Win_datatype_basic(origin_count, - origin_datatype, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count, - target_datatype, - &req->target.dt); - } - - MPID_assert(req->origin.dt.size == req->target.dt.size); - - if ( (req->origin.dt.size == 0) || - (target_rank == MPI_PROC_NULL)) - { - if(req->req_handle) - MPIR_cc_set(req->req_handle->cc_ptr, 0); - else - MPL_free(req); - return MPI_SUCCESS; - } - - req->target.rank = target_rank; - - - if (req->origin.dt.contig) - { - req->buffer_free = 0; - req->buffer = (void *) ((uintptr_t) origin_addr + req->origin.dt.true_lb); - } - else - { - req->buffer_free = 1; - req->buffer = MPL_malloc(req->origin.dt.size); - MPID_assert(req->buffer != NULL); - int mpi_errno = 0; - mpi_errno = MPIR_Localcopy(origin_addr, - origin_count, - origin_datatype, - req->buffer, - req->origin.dt.size, - MPI_CHAR); - MPID_assert(mpi_errno == MPI_SUCCESS); - } - - - pami_result_t rc; - pami_task_t task = MPID_VCR_GET_LPID(win->comm_ptr->vcr, target_rank); - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_START && - !MPIDI_valid_group_rank(task, win->mpid.sync.sc.group)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &req->dest); - MPID_assert(rc == PAMI_SUCCESS); - - - MPIDI_Win_datatype_map(&req->target.dt); - - if (win->create_flavor == MPI_WIN_FLAVOR_SHARED) - { - MPI_User_function *uop; - void *base, *dest_addr; - int disp_unit; - int len, one; - - ++win->mpid.sync.total; - base = win->mpid.info[target_rank].base_addr; - disp_unit = win->mpid.info[target_rank].disp_unit; - dest_addr = (char *) base + disp_unit * target_disp; - - MPIDU_Datatype_get_size_macro(origin_datatype, len); - - uop = MPIR_OP_HDL_TO_FN(op); - one = 1; - - (*uop)((void *) origin_addr, dest_addr, &one, &origin_datatype); - - - MPL_free(req); - ++win->mpid.sync.complete; - - } else { /* non-shared */ - { - win->mpid.sync.total += req->target.dt.num_contig; - MPI_Datatype basic_type = MPI_DATATYPE_NULL; - MPIDU_Datatype_get_basic_type(origin_datatype, basic_type); - /* MPIDU_Datatype_get_basic_type() doesn't handle the struct types */ - if ((origin_datatype == MPI_FLOAT_INT) || - (origin_datatype == MPI_DOUBLE_INT) || - (origin_datatype == MPI_LONG_INT) || - (origin_datatype == MPI_SHORT_INT) || - (origin_datatype == MPI_LONG_DOUBLE_INT)) - { - MPID_assert(basic_type == MPI_DATATYPE_NULL); - basic_type = origin_datatype; - } - MPID_assert(basic_type != MPI_DATATYPE_NULL); - - unsigned index; - MPIDI_Win_MsgInfo * headers = MPL_calloc0(req->target.dt.num_contig, MPIDI_Win_MsgInfo); - req->accum_headers = headers; - for (index=0; index < req->target.dt.num_contig; ++index) { - headers[index].addr = win->mpid.info[target_rank].base_addr + req->offset + - (size_t)req->target.dt.map[index].DLOOP_VECTOR_BUF; - headers[index].req = req; - headers[index].win = win; - headers[index].type = basic_type; - headers[index].op = op; - } - - } - - /* The pamid one-sided design requires context post in order to handle the - * case where the number of pending rma operation exceeds the - * 'PAMID_RMA_PENDING' threshold. When there are too many pending requests the - * work function remains on the context post queue (by returning PAMI_EAGAIN) - * so that the next time the context is advanced the work function will be - * invoked again. - * - * TODO - When context post is not required it would be better to attempt a - * direct context operation and then fail over to using context post if - * the rma pending threshold has been reached. This would result in - * better latency for one-sided operations. - */ - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Accumulate, req); - - } -fn_fail: - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_allocate.c b/src/mpid/pamid/src/onesided/mpid_win_allocate.c deleted file mode 100644 index f42e3a3..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_allocate.c +++ /dev/null @@ -1,99 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_create.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_allocate function - * - * Create a window object. Allocates a MPIR_Win object and initializes it, - * then allocates the collective info array, initalizes our entry, and - * performs an Allgather to distribute/collect the rest of the array entries. - * On each process, it allocates memory of at least size bytes, returns a - * pointer to it, and returns a window object that can be used by all processes - * in comm to * perform RMA operations. The returned memory consists of size - * bytes local to each process, starting at address base_ptr and is associated - * with the window as if the user called 'MPI_Win_create' on existing memory. - * The size argument may be different at each process and size = 0 is valid; - * however, a library might allocate and expose more memory in order to create - * a fast, globally symmetric allocation. - * Input Parameters: - * \param[in] size size of window in bytes (nonnegative integer) - * \param[in] disp_unit local unit size for displacements, in bytes (positive integer) - * \param[in] info info argument (handle)) - * \param[in] comm_ptr Communicator (handle) - * \param[out] base_ptr - base address of the window in local memory - * \param[out] win_ptr window object returned by the call (handle) - * \return MPI_SUCCESS, MPI_ERR_ARG, MPI_ERR_COMM, MPI_ERR_INFO. MPI_ERR_OTHER, - * MPI_ERR_SIZE - */ -int -MPID_Win_allocate(MPI_Aint size, - int disp_unit, - MPIR_Info * info, - MPIR_Comm * comm_ptr, - void *base_ptr, - MPIR_Win ** win_ptr) -{ - int mpi_errno = MPI_SUCCESS; - int rc = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - void *baseP; - static char FCNAME[] = "MPID_Win_allocate"; - MPIDI_Win_info *winfo; - MPIR_Win *win; - int rank; - - rc=MPIDI_Win_init(size,disp_unit,win_ptr, info, comm_ptr, MPI_WIN_FLAVOR_ALLOCATE, MPI_WIN_UNIFIED); - win = *win_ptr; - - if (size > 0) { - baseP = MPL_malloc(size); - #ifndef MPIDI_NO_ASSERT - MPID_assert(baseP != NULL); - #else - MPIU_ERR_CHKANDJUMP((baseP == NULL), mpi_errno, MPI_ERR_BUFFER, "**bufnull"); - #endif - - } else if (size == 0) { - baseP = NULL; - } else { - MPIU_ERR_CHKANDSTMT(size >=0 , mpi_errno, MPI_ERR_SIZE, - return mpi_errno, "**rmasize"); - } - - win->base = baseP; - rank = comm_ptr->rank; - winfo = &win->mpid.info[rank]; - winfo->base_addr = baseP; - winfo->win = win; - winfo->disp_unit = disp_unit; - - rc= MPIDI_Win_allgather(size,win_ptr); - if (rc != MPI_SUCCESS) - return rc; - *(void**) base_ptr = (void *) win->base; - mpi_errno = MPIR_Barrier_impl(comm_ptr, &errflag); - - fn_fail: - return mpi_errno; -} - diff --git a/src/mpid/pamid/src/onesided/mpid_win_allocate_shared.c b/src/mpid/pamid/src/onesided/mpid_win_allocate_shared.c deleted file mode 100644 index dcafece..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_allocate_shared.c +++ /dev/null @@ -1,561 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_allocate_shared.c - * \brief - */ -#include "mpidi_onesided.h" -#include <sys/shm.h> -#include <sys/ipc.h> -#include <sys/stat.h> -#include <fcntl.h> -#include <sys/mman.h> - -#undef FUNCNAME -#define FUNCNAME MPID_Win_allocate_shared -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) - -#define SHM_KEY_TAIL 0xfdbc /* If this value is changed, change */ - /* in PMD (mp_pmd.c) as well. */ - -extern int mpidi_dynamic_tasking; -#define MPIDI_PAGESIZE_MASK(y) (~(((MPI_Aint)y)-1)) -#define MPIDI_ROUND_UP_PAGESIZE(x,y) ((((MPI_Aint)x)+(~MPIDI_PAGESIZE_MASK(y))) & MPIDI_PAGESIZE_MASK(y)) -#define ALIGN_BOUNDARY 128 /* Align data structures to cache line */ -#define PAD_SIZE(s) (ALIGN_BOUNDARY - (sizeof(s) & (ALIGN_BOUNDARY-1))) - -int CheckRankOnNode(MPIR_Comm * comm_ptr,int *onNode ) { - int comm_size, i; - int mpi_errno = PAMI_SUCCESS; - - comm_size = comm_ptr->local_size; - - *onNode = 1; - - for (i = 0; i < comm_size; i++) { - if (comm_ptr->intranode_table[i] == -1) { - *onNode = 0; - break; - } - } - - if (*onNode== 0) { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_CONFLICT, - return mpi_errno, "**rmaconflict"); - } - - return mpi_errno; -} - -int CheckSpaceType(MPIR_Win **win_ptr, MPIR_Info *info,int *noncontig) { - int mpi_errno=MPI_SUCCESS; - /* Check if we are allowed to allocate space non-contiguously */ - if (info != NULL) { - int alloc_shared_nctg_flag = 0; - char alloc_shared_nctg_value[MPI_MAX_INFO_VAL+1]; - MPIR_Info_get_impl(info, "alloc_shared_noncontig", MPI_MAX_INFO_VAL, - alloc_shared_nctg_value, &alloc_shared_nctg_flag); - if (alloc_shared_nctg_flag == 1) { - if (!strncmp(alloc_shared_nctg_value, "true", strlen("true"))) - (*win_ptr)->mpid.info_args.alloc_shared_noncontig = 1; - if (!strncmp(alloc_shared_nctg_value, "false", strlen("false"))) - (*win_ptr)->mpid.info_args.alloc_shared_noncontig = 0; - } - } - *noncontig = (*win_ptr)->mpid.info_args.alloc_shared_noncontig; - return mpi_errno; -} - -int GetPageSize(void *addr, ulong *pageSize) -{ - pid_t pid; - FILE *fp; - char Line[201],A1[50],A2[50]; - char fileName[100]; - char A3[50],A4[50]; - int i=0; - char *t1,*t2; - #ifndef REDHAT - char a='-'; - char *search = &a; - #endif - void *beg, *end; - int found=0; - int ps; - int j,k; - - *pageSize = 0; - pid = getpid(); - sprintf(fileName,"/proc/%d/smaps",pid); - /* linux-2.6.29 or greater, KernelPageSize in /proc/pid/smaps includes 4K, 16 MB */ - TRACE_ERR("fileName = %s addr=%p\n",fileName,addr); - fp = fopen(fileName,"r"); - if (fp == NULL) { - TRACE_ERR("fileName = %s open failed errno=%d\n",fileName,errno); - return errno; - } - while(fgets(Line,200,fp)) { - i++; - sscanf(Line,"%s %s %s %s \n",A1,A2,A3,A4); - if ((found == 1) && (memcmp(A1,"KernelPageSize",14)==0)) { - j=atoi(A2); - if ((A3[0]=='k') || (A3[0]=='K')) - k=1024; - else if ((A3[0]=='m') || (A3[0]=='M')) - k=1048576; - else if ((A3[0]=='g') || (A3[0]=='G')) - k=0x40000000; /* 1 GB */ - else { - TRACE_ERR("ERROR unrecognized unit A3=%s\n",A3); - break; - } - *pageSize = (ulong)(j * k); - TRACE_ERR(" addr=%p pageSize=%ld %s(%d)\n", addr,*pageSize,__FILE__,__LINE__); - break; - } - if ((strlen(A2) == 4) && ((A2[0]=='r') || (A2[3]=='p'))) { - #ifndef REDHAT - t1=strtok(A1,search); - #else - t1=strtok(A1,"-"); - #endif - t2 = A1+strlen(t1)+1; - sscanf(t1,"%p \n",&beg); - sscanf(t2,"%p \n",&end); - if (((ulong) addr >= (ulong)beg) && ((ulong)addr <= (ulong)end)) { - found=1; - TRACE_ERR("found addr=%p i=%d between beg=%p and end=%p in %s\n", - addr,i,beg,end,fileName); - } - } - } - fclose(fp); - if (*pageSize == 0) { - ps = getpagesize(); - *pageSize = (ulong) ps; - TRACE_ERR("LinuxPageSize %p not in %s getpagesize=%ld\n", addr,fileName,*pageSize); - } - return 0; -} - -int -MPID_getSharedSegment_mmap(MPIR_Win * win) -{ - int rank, rc, fd; - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - int first = 0; - - snprintf (win->mpid.shm->shm_key, 63, "/mpich.comm-%d.win_shared", win->comm_ptr->context_id); - rc = shm_open (win->mpid.shm->shm_key, O_RDWR | O_CREAT | O_EXCL, 0600); - if (0 == rc) - { - first = 1; - } else { - rc = shm_open (win->mpid.shm->shm_key, O_RDWR, 0); - MPIU_ERR_CHKANDJUMP((rc == -1), mpi_errno, MPI_ERR_RMA_SHARED, "**rmashared"); - } - - fd = rc; - rc = ftruncate (fd, win->mpid.shm->segment_len); - MPIU_ERR_CHKANDJUMP((rc == -1), mpi_errno, MPI_ERR_RMA_SHARED, "**rmashared"); - - win->mpid.shm->base_addr = mmap (NULL, win->mpid.shm->segment_len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); - MPIU_ERR_CHKANDJUMP((win->mpid.shm->base_addr == MAP_FAILED), mpi_errno, MPI_ERR_RMA_SHARED, "**rmashared"); - - close (fd); /* no longer needed */ - - /* set mutex_lock address and initialize it */ - win->mpid.shm->ctrl = (MPIDI_Win_shm_ctrl_t *) win->mpid.shm->base_addr; - if (1 == first) { - MPIDI_SHM_MUTEX_INIT(win); - } - - mpi_errno = MPIR_Barrier_impl(win->comm_ptr, &errflag); - MPIU_ERR_CHKANDJUMP(errflag, mpi_errno, MPI_ERR_OTHER, "**coll_fail"); - - win->mpid.shm->allocated = 1; - -fn_exit: - return mpi_errno; - /* --BEGIN ERROR HANDLING-- */ -fn_fail: - shm_unlink (win->mpid.shm->shm_key); - goto fn_exit; - /* --END ERROR HANDLING-- */ -} - -int -MPID_getSharedSegment_sysv(MPIR_Win * win) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - uint32_t shm_key; - int rank; - char *cp; - int shm_flag = IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR; - - shm_key = (uint32_t) -1; - - rank = win->comm_ptr->rank; - - if (rank == 0) { -#ifdef DYNAMIC_TASKING - /* generate an appropriate key */ - if (!mpidi_dynamic_tasking) { - cp = getenv("MP_I_PMD_PID"); - if (cp) { - shm_key = atoi(cp); - shm_key = shm_key & 0x07ffffff; - shm_key = shm_key | 0x80000000; - } else { - cp = getenv("MP_PARTITION"); - if (cp ) { - shm_key = atol(cp); - shm_key = (shm_key << 16) + SHM_KEY_TAIL; - } else { - TRACE_ERR("ERROR MP_PARTITION not set \n"); - } - } - } else { - cp = getenv("MP_I_KEY_RANGE"); - if (cp) { - sscanf(cp, "0x%x", &shm_key); - shm_key = shm_key | 0x80; - } else { - TRACE_ERR("ERROR MP_I_KEY_RANGE not set \n"); - } - } -#else - cp = getenv("MP_I_PMD_PID"); - if (cp) { - shm_key = atoi(cp); - shm_key = shm_key & 0x07ffffff; - shm_key = shm_key | 0x80000000; - } else { - cp = getenv("MP_PARTITION"); - if (cp ) { - shm_key = atol(cp); - shm_key = (shm_key << 16); -#ifdef SHMCC_KEY_TAIL - shm_key += SHMCC_KEY_TAIL; -#endif - } else { - TRACE_ERR("ERROR MP_PARTITION not set \n"); - } - } -#endif - - MPID_assert(shm_key != -1); - - win->mpid.shm->shm_id = shmget(shm_key, win->mpid.shm->segment_len, shm_flag); - MPIU_ERR_CHKANDJUMP((win->mpid.shm->shm_id == -1), mpi_errno, MPI_ERR_RMA_SHARED, "**rmashared"); - - win->mpid.shm->base_addr = (void *) shmat(win->mpid.shm->shm_id,0,0); - MPIU_ERR_CHKANDJUMP((win->mpid.shm->base_addr == (void*) -1), mpi_errno,MPI_ERR_BUFFER, "**bufnull"); - - /* set mutex_lock address and initialize it */ - win->mpid.shm->ctrl = (MPIDI_Win_shm_ctrl_t *) win->mpid.shm->base_addr; - MPIDI_SHM_MUTEX_INIT(win); - - /* successfully created shm segment - shared the key with other tasks */ - mpi_errno = MPIR_Bcast_impl((void *) &shm_key, sizeof(int), MPI_CHAR, 0, win->comm_ptr, &errflag); - - } else { /* task other than task 0 */ - mpi_errno = MPIR_Bcast_impl((void *) &shm_key, sizeof(int), MPI_CHAR, 0, win->comm_ptr, &errflag); - MPIU_ERR_CHKANDJUMP(errflag, mpi_errno, MPI_ERR_OTHER, "**coll_fail"); - - win->mpid.shm->shm_id = shmget(shm_key, 0, 0); - if (win->mpid.shm->shm_id != -1) { /* shm segment is available */ - win->mpid.shm->base_addr = (void *) shmat(win->mpid.shm->shm_id,0,0); - } - win->mpid.shm->ctrl = (MPIDI_Win_shm_ctrl_t *) win->mpid.shm->base_addr; - } - - win->mpid.shm->allocated = 1; - -fn_exit: - return mpi_errno; - /* --BEGIN ERROR HANDLING-- */ -fn_fail: - goto fn_exit; - /* --END ERROR HANDLING-- */ -} - -int -MPID_getSharedSegment(MPI_Aint size, - int disp_unit, - MPIR_Comm * comm_ptr, - MPIR_Win **win_ptr, - MPI_Aint * pSize, - int * noncontig) -{ - int mpi_errno = MPI_SUCCESS; - int i, comm_size, rank; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - MPI_Aint pageSize,pageSize2, len,new_size; - MPIR_Win *win; - int padSize; - void *base_pp; - - win = *win_ptr; - comm_size = win->comm_ptr->local_size; - rank = win->comm_ptr->rank; - - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - GetPageSize((void *) win_ptr, (ulong *) &pageSize); - *pSize = pageSize; - win->mpid.shm->segment_len = 0; - - if (comm_size == 1) { - /* Do not use shared memory when there is only one rank on the node */ - - /* 'size' must not be < 0 */ - MPIU_ERR_CHKANDSTMT(size < 0 , mpi_errno, MPI_ERR_SIZE,return mpi_errno, "**rmasize"); - - /* The beginning of the heap allocation contains a control block - * before the data begins. - */ - new_size = MPIDI_ROUND_UP_PAGESIZE((sizeof(MPIDI_Win_shm_ctrl_t)+ ((comm_size+1) * sizeof(void *))),pageSize); - - if (size > 0) { - if (*noncontig) - new_size += MPIDI_ROUND_UP_PAGESIZE(size,pageSize); - else - new_size += size; - } - - base_pp = MPL_malloc(new_size); - MPID_assert(base_pp !=NULL); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - win->mpid.shm->segment_len = new_size; - win->mpid.shm->base_addr = base_pp; - if (size !=0) { - win->mpid.info[rank].base_addr = (void *)((MPI_Aint) base_pp + MPIDI_ROUND_UP_PAGESIZE((sizeof(MPIDI_Win_shm_ctrl_t) + ((comm_size+1) * sizeof(void *))),pageSize)); - } else { - win->mpid.info[rank].base_addr = NULL; - } - win->base = win->mpid.info[rank].base_addr; - win->mpid.info[rank].base_size = size; - - /* set mutex_lock address and initialize it */ - win->mpid.shm->ctrl = (MPIDI_Win_shm_ctrl_t *) win->mpid.shm->base_addr; - MPIDI_SHM_MUTEX_INIT(win); - OPA_fetch_and_add_int((OPA_int_t *) &win->mpid.shm->ctrl->shm_count,(int) 1); - - } else { - /* allocate a temporary buffer to gather the 'size' of each buffer on - * the node to determine the amount of shared memory to allocate - */ - MPI_Aint * size_array; - size_array = MPL_malloc (2*comm_size*sizeof(MPI_Aint)); - size_array[rank] = (MPI_Aint) size; - mpi_errno = MPIR_Allgather_impl(MPI_IN_PLACE, 0, MPI_DATATYPE_NULL, - size_array, 1 * sizeof(MPI_Aint), MPI_BYTE, - (*win_ptr)->comm_ptr, &errflag); - if (mpi_errno) { - MPL_free(size_array); - MPIU_ERR_POP(mpi_errno); - } - - /* calculate total number of bytes needed */ - MPI_Aint actual_size; - win->mpid.info[0].base_addr = NULL; - for (i = 0; i < comm_size; ++i) { - win->mpid.info[i].base_size = size_array[i]; - - actual_size = (*noncontig)?MPIDI_ROUND_UP_PAGESIZE(size_array[i],pageSize):size_array[i]; - - win->mpid.shm->segment_len += actual_size; - - /* Save the OFFSET to each rank's private shared memory area. This - * will be added to the BASE ADDRESS of the entire shared memory - * allocation to determine the virtual address. - */ - if (i < comm_size-1) - win->mpid.info[i+1].base_addr = - (void *) ((uintptr_t)win->mpid.info[i].base_addr + actual_size); - } - - /* The beginning of the shared memory allocation contains a control - * block before the data begins. - */ - win->mpid.shm->segment_len += MPIDI_ROUND_UP_PAGESIZE((sizeof(MPIDI_Win_shm_ctrl_t) + ((comm_size+1) * sizeof(void *))),pageSize); - - /* Get the shared segment which includes the control block header and - * data buffer - possibly padded if non-contiguous. - */ -#ifdef USE_SYSV_SHM - mpi_errno = MPID_getSharedSegment_sysv(win); -#elif USE_MMAP_SHM - mpi_errno = MPID_getSharedSegment_mmap(win); -#else - MPID_Abort(NULL, MPI_ERR_RMA_SHARED, -1, "RMA shared segment error"); -#endif - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - /* assign base address here */ - /* compute the base addresses of each process within the shared memory segment */ - /* will be done after MPI_Allgather in MID_Win_allocate_shared */ - win->base = (void *) ((long) win->mpid.shm->base_addr + (long ) MPIDI_ROUND_UP_PAGESIZE((sizeof(MPIDI_Win_shm_ctrl_t) + ((comm_size+1) * sizeof(void *))),pageSize)); - - - MPL_free(size_array); - - /* increment the shared counter */ - OPA_fetch_and_add_int((OPA_int_t *) &win->mpid.shm->ctrl->shm_count,(int) 1); - - /* wait for all ranks complete */ - while((int) win->mpid.shm->ctrl->shm_count != comm_size) MPIDI_QUICKSLEEP; - } - -fn_exit: - return mpi_errno; - /* --BEGIN ERROR HANDLING-- */ -fn_fail: - goto fn_exit; - /* --END ERROR HANDLING-- */ - -} - -/** - * \brief MPI-PAMI glue for MPI_Win_allocate_shared function - * - * Create a window object. Allocates a MPIR_Win object and initializes it, - * then allocates the collective info array, initalizes our entry, and - * performs an Allgather to distribute/collect the rest of the array entries. - * On each process, it allocates memory of at least size bytes that is shared - * among all processes in comm, and returns a pointer to the locally allocated - * segment in base_ptr that can be used for load/store accesses on the calling - * process. The locally allocated memory can be the target of load/store accessses - * by remote processes; the base pointers for other processes can be queried using - * the function 'MPI_Win_shared_query'. - * - * The call also returns a window object that can be used by all processes in comm - * to perform RMA operations. The size argument may be different at each process - * and size = 0 is valid. It is the user''s responsibility to ensure that the - * communicator comm represents a group of processes that can create a shared - * memory segment that can be accessed by all processes in the group. The - * allocated memory is contiguous across process ranks unless the info key - * alloc_shared_noncontig is specified. Contiguous across process ranks means that - * the first address in the memory segment of process i is consecutive with the - * last address in the memory segment of process i - 1. This may enable the user - * to calculate remote address offsets with local information only. - * - * Input Parameters: - * \param[in] size size of window in bytes (nonnegative integer) - * \param[in] disp_unit local unit size for displacements, in bytes (positive integer) - * \param[in] info info argument (handle)) - * \param[in] comm_ptr intra-Communicator (handle) - * \param[out] base_ptr address of local allocated window segment - * \param[out] win_ptr window object returned by the call (handle) - * \return MPI_SUCCESS, MPI_ERR_ARG, MPI_ERR_COMM, MPI_ERR_INFO. MPI_ERR_OTHER, - * MPI_ERR_SIZE - * - * win->mpid.shm->base_addr \* return address from shmat *\ - * win->base \* address for data starts here == win->mpid.shm->base_addr *\ - * \* + space for mutex_lock and shm_count *\ - * - */ -int -MPID_Win_allocate_shared(MPI_Aint size, - int disp_unit, - MPIR_Info * info, - MPIR_Comm * comm_ptr, - void *base_ptr, - MPIR_Win ** win_ptr) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - int onNode = 0; - MPIR_Win *win = NULL; - int rank, prev_size; - - MPIDI_Win_info *winfo; - int comm_size,i; - int noncontig=FALSE; - MPI_Aint pageSize=0; - - /* Verify all ranks are on-node */ - mpi_errno=CheckRankOnNode(comm_ptr,&onNode); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - MPIU_ERR_CHKANDJUMP((onNode == 0), mpi_errno, MPI_ERR_RMA_SHARED, "**rmashared"); - - /* Initialize the window */ - mpi_errno =MPIDI_Win_init(size,disp_unit,win_ptr, info, comm_ptr, MPI_WIN_FLAVOR_SHARED, MPI_WIN_UNIFIED); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - win = *win_ptr; - win->mpid.shm = MPL_malloc(sizeof(MPIDI_Win_shm_t)); - MPID_assert(win->mpid.shm != NULL); - memset(win->mpid.shm, 0, sizeof(MPIDI_Win_shm_t)); - - rank = comm_ptr->rank; - win->mpid.info[rank].win = win; - win->mpid.info[rank].disp_unit = disp_unit; - - mpi_errno=CheckSpaceType(win_ptr,info,&noncontig); - comm_size = (*win_ptr)->comm_ptr->local_size; - mpi_errno = MPID_getSharedSegment(size, disp_unit,comm_ptr, win_ptr, &pageSize, &noncontig); - if (mpi_errno != MPI_SUCCESS) - return mpi_errno; - - mpi_errno = MPIDI_Win_allgather(size,win_ptr); - if (mpi_errno != MPI_SUCCESS) - return mpi_errno; - - win->mpid.info[0].base_addr = win->base; - char *cur_base = win->base; - prev_size=win->mpid.info[0].base_size; - for (i = 1; i < comm_size; ++i) { - if (win->mpid.info[i].base_size) { - if (prev_size) { - if (noncontig) { - /* Round up to next page size */ - win->mpid.info[i].base_addr = - (void *) ((MPI_Aint) cur_base + (MPI_Aint) MPIDI_ROUND_UP_PAGESIZE(prev_size,pageSize)); - } else { - win->mpid.info[i].base_addr = (void *) ((MPI_Aint) cur_base + (MPI_Aint) prev_size); - } - } else - win->mpid.info[i].base_addr = (void *) ((MPI_Aint) cur_base); - prev_size=win->mpid.info[i].base_size; - cur_base = win->mpid.info[i].base_addr; - } else { - win->mpid.info[i].base_addr = NULL; - } - } - - - *(void**) base_ptr = (void *) win->mpid.info[rank].base_addr; - - mpi_errno = MPIR_Barrier_impl(comm_ptr, &errflag); -fn_exit: - return mpi_errno; - /* --BEGIN ERROR HANDLING-- */ -fn_fail: - if (win != NULL) - if (win->mpid.shm != NULL) - MPL_free(win->mpid.shm); - goto fn_exit; - /* --END ERROR HANDLING-- */ - -} - diff --git a/src/mpid/pamid/src/onesided/mpid_win_attach.c b/src/mpid/pamid/src/onesided/mpid_win_attach.c deleted file mode 100644 index aae21a4..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_attach.c +++ /dev/null @@ -1,58 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_attach.c - * \brief attaches a local memory region beginning at base for remote - * access within the given window. - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_attach function - * - * Attaches a local memory region beginning at base for remote access - * within the given window. - * - * \param[in] win shared memory window object - * \param[in] base initial address of memory to be attached - * \param[in] length length of memory to be attached in bytes - * \return MPI_SUCCESS, MPI_ERR_RMA_FLAVOR - * - */ - -int -MPID_Win_attach(MPIR_Win *win, void *base, MPI_Aint size) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_attach"; - MPIU_ERR_CHKANDSTMT((win->create_flavor != MPI_WIN_FLAVOR_DYNAMIC), mpi_errno, - MPI_ERR_RMA_FLAVOR, return mpi_errno, "**rmaflavor"); - - - /* no op, all memory is exposed, the user is responsible for */ - /* ensuring that MPI_WIN_ATTACH at the target has returned */ - /* before a process attempts to target that memory with an */ - /* RMA call */ - - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - diff --git a/src/mpid/pamid/src/onesided/mpid_win_compare_and_swap.c b/src/mpid/pamid/src/onesided/mpid_win_compare_and_swap.c deleted file mode 100644 index bfb48a0..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_compare_and_swap.c +++ /dev/null @@ -1,164 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_compare_and_swap.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -extern pami_result_t -MPIDI_Atomic (pami_context_t context, - void * _req); - -#ifndef __BGQ__ -static pami_result_t -MPIDI_Compare_and_swap_using_pami_rmw(pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - int target_rank; - - MPID_assert(req != NULL); - target_rank = req->target.rank; - - pami_rmw_t params; - params=zero_rmw_parms; - params.dest=req->dest; - params.cookie=(void *)req; - params.done_fn=MPIDI_Win_DoneCB; - params.type = req->pami_datatype; - params.operation = PAMI_ATOMIC_FETCH_COMPARE_SET; - params.local=req->user_buffer; /*result*/ - params.remote=req->win->mpid.info[target_rank].base_addr + req->offset + (size_t)req->origin.dt.map[0].DLOOP_VECTOR_BUF; - params.value=req->buffer; /* replaced value with origin */ - params.test=req->compare_buffer; - - rc = PAMI_Rmw(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return rc; -} -#endif - -#undef FUNCNAME -#define FUNCNAME MPIDI_Compare_and_swap -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_Compare_and_swap(const void *origin_addr, const void *compare_addr, - void *result_addr, MPI_Datatype datatype, int target_rank, - MPI_Aint target_disp, MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - MPIDI_Win_request *req; - int shm_locked=0; - - if(win->mpid.sync.origin_epoch_type == win->mpid.sync.target_epoch_type && - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_FENCE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_FENCE; - } - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_NONE || - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - if (target_rank == MPI_PROC_NULL) - { - return MPI_SUCCESS; - } - /* Check if datatype is a C integer, Fortran Integer, - logical, or byte, per the classes given on page 165. */ - MPIR_ERRTEST_TYPE_RMA_ATOMIC(datatype, mpi_errno); - - req = (MPIDI_Win_request *) MPL_calloc0(1, MPIDI_Win_request); - req->win = win; - req->type = MPIDI_WIN_REQUEST_COMPARE_AND_SWAP; - - req->offset = target_disp * win->mpid.info[target_rank].disp_unit; -#ifdef __BGQ__ - /* PAMI limitation as it doesnt permit VA of 0 to be passed into - * memregion create, so we must pass base_va of heap computed from - * an SPI call instead. So the target offset must be adjusted */ - if (req->win->create_flavor == MPI_WIN_FLAVOR_DYNAMIC) - req->offset -= (size_t)req->win->mpid.info[target_rank].base_addr; -#endif - - MPIDI_Win_datatype_basic(1, datatype, &req->origin.dt); - - if (req->origin.dt.size == 0) - { - MPL_free(req); - return MPI_SUCCESS; - } - - req->target.rank = target_rank; - req->buffer = (void *) ((uintptr_t) origin_addr + req->origin.dt.true_lb); - req->user_buffer = result_addr + req->origin.dt.true_lb; - req->compare_buffer = (void *) ((uintptr_t) compare_addr + req->origin.dt.true_lb); - - pami_result_t rc; - pami_task_t task = MPID_VCR_GET_LPID(win->comm_ptr->vcr, target_rank); - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_START && - !MPIDI_valid_group_rank(task, win->mpid.sync.sc.group)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &req->dest); - MPID_assert(rc == PAMI_SUCCESS); - - MPIDI_Win_datatype_map(&req->origin.dt); - win->mpid.sync.total += 1; - - MPI_Datatype basic_type = MPI_DATATYPE_NULL; - MPIDU_Datatype_get_basic_type(datatype, basic_type); - MPID_assert(basic_type != MPI_DATATYPE_NULL); - req->origin.datatype=basic_type; - - /* The pamid one-sided design requires context post in order to handle the - * case where the number of pending rma operation exceeds the - * 'PAMID_RMA_PENDING' threshold. When there are too many pending requests the - * work function remains on the context post queue (by returning PAMI_EAGAIN) - * so that the next time the context is advanced the work function will be - * invoked again. - * - * TODO - When context post is not required it would be better to attempt a - * direct context operation and then fail over to using context post if - * the rma pending threshold has been reached. This would result in - * better latency for one-sided operations. - */ - -#ifndef __BGQ__ - MPI_Op null_op=0; - pami_data_function pami_op; - pami_type_t pami_type; - if(MPIDI_Datatype_is_pami_rmw_supported(basic_type, &pami_type, null_op, &pami_op) ) { - req->pami_datatype = pami_type; - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Compare_and_swap_using_pami_rmw, req); - } else -#endif - { - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Atomic, req); - } - -fn_fail: - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_create.c b/src/mpid/pamid/src/onesided/mpid_win_create.c deleted file mode 100644 index ae7a062..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_create.c +++ /dev/null @@ -1,213 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_create.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -/***************************************************************************/ -/* */ -/* allocate win_ptr (MPIDI_Win) */ -/* update win structure except for base address */ -/* */ -/***************************************************************************/ - -int -MPIDI_Win_init( MPI_Aint length, - int disp_unit, - MPIR_Win **win_ptr, - MPIR_Info *info, - MPIR_Comm *comm_ptr, - int create_flavor, - int model) -{ - int mpi_errno=MPI_SUCCESS; - size_t rank, size; - MPIDI_Win_info *winfo; - static char FCNAME[] = "MPIDI_Win_init"; - - /* ----------------------------------------- */ - /* Setup the common sections of the window */ - /* ----------------------------------------- */ - MPIR_Win *win = (MPIR_Win*)MPIR_Handle_obj_alloc(&MPIR_Win_mem); - - MPIU_ERR_CHKANDSTMT(win == NULL, mpi_errno, MPI_ERR_NO_MEM, - return mpi_errno, "**nomem"); - - *win_ptr = win; - memset(&win->mpid, 0, sizeof(struct MPIDI_Win)); - win->comm_ptr = comm_ptr; MPIR_Comm_add_ref(comm_ptr); - size = comm_ptr->local_size; - rank = comm_ptr->rank; - - win->mpid.info = MPL_malloc(size * sizeof(struct MPIDI_Win_info)); - MPID_assert(win->mpid.info != NULL); - memset((void *) win->mpid.info,0,(size * sizeof(struct MPIDI_Win_info))); - winfo = &win->mpid.info[rank]; - win->errhandler = NULL; - win->base = NULL; - win->size = length; - win->disp_unit = disp_unit; - win->create_flavor = create_flavor; - win->model = model; - win->copyCreateFlavor = 0; - win->copyModel = 0; - win->attributes = NULL; - win->comm_ptr = comm_ptr; - if ((info != NULL) && ((int *)info != (int *) MPI_INFO_NULL)) { - mpi_errno= MPIDI_Win_set_info(win, info); - MPID_assert(mpi_errno == 0); - } - MPID_assert(mpi_errno == 0); - - - /* Initialize the info (hint) flags per window */ - win->mpid.info_args.no_locks = 0; - win->mpid.info_args.accumulate_ordering = - (MPIDI_ACCU_ORDER_RAR | MPIDI_ACCU_ORDER_RAW | MPIDI_ACCU_ORDER_WAR | MPIDI_ACCU_ORDER_WAW); - win->mpid.info_args.accumulate_ops = MPIDI_ACCU_SAME_OP_NO_OP; /*default */ - win->mpid.info_args.same_size = 0; - win->mpid.info_args.alloc_shared_noncontig = 0; - - win->copyDispUnit=0; - win->copySize=0; - winfo->memregion_used = 0; - winfo->disp_unit = disp_unit; - - return mpi_errno; -} - -/***************************************************************************/ -/* */ -/* MPIDI_Win_allgather */ -/* */ -/* registers memory with PAMI if possible */ -/* calls Allgather to gather the information from all members in win. */ -/* */ -/***************************************************************************/ -int -MPIDI_Win_allgather( MPI_Aint size, MPIR_Win **win_ptr ) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag=MPIR_ERR_NONE; - MPIR_Win *win; - int rank; - MPIR_Comm *comm_ptr; - size_t length_out = 0; - pami_result_t rc; - MPIDI_Win_info *winfo; - static char FCNAME[] = "MPIDI_Win_allgather"; - - win = *win_ptr; - comm_ptr = win->comm_ptr; - rank = comm_ptr->rank; - winfo = &win->mpid.info[rank]; - - if (size != 0 && win->create_flavor != MPI_WIN_FLAVOR_SHARED) - { -#ifndef USE_PAMI_RDMA - if (!MPIDI_Process.mp_s_use_pami_get) - { -#endif - /* --------------------------------------- */ - /* Setup the PAMI sections of the window */ - /* --------------------------------------- */ - rc = PAMI_Memregion_create(MPIDI_Context[0], win->mpid.info[rank].base_addr, win->size, &length_out, &winfo->memregion); -#ifdef USE_PAMI_RDMA - MPIU_ERR_CHKANDJUMP((rc != PAMI_SUCCESS), mpi_errno, MPI_ERR_OTHER, "**nomem"); - MPIU_ERR_CHKANDJUMP((win->size < length_out), mpi_errno, MPI_ERR_OTHER, "**nomem"); -#else - if (rc == PAMI_SUCCESS) - { - winfo->memregion_used = 1; - MPID_assert(win->size == length_out); - } - } -#endif - } - - mpi_errno = MPIR_Allgather_impl(MPI_IN_PLACE, - 0, - MPI_DATATYPE_NULL, - win->mpid.info, - sizeof(struct MPIDI_Win_info), - MPI_BYTE, - comm_ptr, - &errflag); - -fn_fail: - return mpi_errno; -} - - - -/** - * \brief MPI-PAMI glue for MPI_Win_create function - * - * Create a window object. Allocates a MPIR_Win object and initializes it, - * then allocates the collective info array, initalizes our entry, and - * performs an Allgather to distribute/collect the rest of the array entries. - * - * ON first call, initializes (registers) protocol objects for locking, - * get, and send operations to message layer. Also creates datatype to - * represent the rma_sends element of the collective info array, - * used later to synchronize epoch end events. - * - * \param[in] base Local window buffer - * \param[in] size Local window size - * \param[in] disp_unit Displacement unit size - * \param[in] info Window hints (not used) - * \param[in] comm_ptr Communicator - * \param[out] win_ptr Window - * \return MPI_SUCCESS, MPI_ERR_OTHER, or error returned from - * MPI_Comm_dup or MPI_Allgather. - */ -int -MPID_Win_create(void * base, - MPI_Aint size, - int disp_unit, - MPIR_Info * info, - MPIR_Comm * comm_ptr, - MPIR_Win ** win_ptr) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - int rc = MPI_SUCCESS; - MPIR_Win *win; - size_t rank; - MPIDI_Win_info *winfo; - - rc=MPIDI_Win_init(size,disp_unit,win_ptr, info, comm_ptr, MPI_WIN_FLAVOR_CREATE, MPI_WIN_UNIFIED); - win = *win_ptr; - win->base = base; - rank = comm_ptr->rank; - winfo = &win->mpid.info[rank]; - winfo->base_addr = base; - winfo->win = win; - winfo->disp_unit = disp_unit; - - rc= MPIDI_Win_allgather(size,win_ptr); - if (rc != MPI_SUCCESS) - return rc; - - - mpi_errno = MPIR_Barrier_impl(comm_ptr, (MPIR_Errflag_t *) &errflag); - - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_create_dynamic.c b/src/mpid/pamid/src/onesided/mpid_win_create_dynamic.c deleted file mode 100644 index 6b989ac..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_create_dynamic.c +++ /dev/null @@ -1,90 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_create.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_create_dynamic function - * - * Create a window object. Allocates a MPIR_Win object and initializes it, - * then allocates the collective info array, initalizes our entry, and - * performs an Allgather to distribute/collect the rest of the array entries. - * The function returns a window win without memory attached. - * - * Input Parameters: - * \param[in] info info argument - * \param[in] comm intra-Communicator (handle) - * \param[out] win_ptr window object returned by the call (handle) - * \return MPI_SUCCESS, MPI_ERR_ARG, MPI_ERR_COMM, MPI_ERR_INFO. MPI_ERR_OTHER, - * MPI_ERR_SIZE - */ - -int -MPID_Win_create_dynamic( MPIR_Info * info, - MPIR_Comm * comm_ptr, - MPIR_Win ** win_ptr) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - int rc = MPI_SUCCESS; - MPIDI_Win_info *winfo; - MPIR_Win *win; - int rank; - - rc=MPIDI_Win_init(0,1,win_ptr, info, comm_ptr, MPI_WIN_FLAVOR_DYNAMIC, MPI_WIN_UNIFIED); - win = *win_ptr; - - rank = comm_ptr->rank; - win->base = MPI_BOTTOM; - winfo = &win->mpid.info[rank]; - winfo->win = win; - -#ifdef __BGQ__ - void *base = NULL; - size_t length = 0; - Kernel_MemoryRegion_t memregion; - void *tmpbuf = MPL_malloc(sizeof(int)); - Kernel_CreateMemoryRegion(&memregion, tmpbuf, sizeof(int)); - //Reset base to base VA of local heap - base = memregion.BaseVa; - length = memregion.Bytes; - MPL_free(tmpbuf); - - if (length != 0) - { - size_t length_out = 0; - pami_result_t rc; - rc = PAMI_Memregion_create(MPIDI_Context[0], base, length, &length_out, &winfo->memregion); - MPID_assert(rc == PAMI_SUCCESS); - MPID_assert(length == length_out); - } - win->base = base; - winfo->base_addr = base; -#endif - - rc= MPIDI_Win_allgather(0,win_ptr); - if (rc != MPI_SUCCESS) - return rc; - - mpi_errno = MPIR_Barrier_impl(comm_ptr, (MPIR_Errflag_t *) &errflag); - - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_detach.c b/src/mpid/pamid/src/onesided/mpid_win_detach.c deleted file mode 100644 index 4a84052..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_detach.c +++ /dev/null @@ -1,53 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_detach.c - * \brief detaches a previously attached memory region beginning at - * base - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_detach function - * - * Detaches a previously attached memory beginning at base. - * The arguments base and win must match the arguments passed - * to a previous call to MPI_Win_attach. - * \param[in] win window object - * \param[in] base initial address of emmory to be detached - * \return MPI_SUCCESS, MPI_ERR_RMA_FLAVOR - * - */ - -int -MPID_Win_detach(MPIR_Win *win, const void *base) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_detach"; - MPIU_ERR_CHKANDSTMT((win->create_flavor != MPI_WIN_FLAVOR_DYNAMIC), mpi_errno, - MPI_ERR_RMA_FLAVOR, return mpi_errno, "**rmaflavor"); - - - /* no op, all memory is exposed */ - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - diff --git a/src/mpid/pamid/src/onesided/mpid_win_fence.c b/src/mpid/pamid/src/onesided/mpid_win_fence.c deleted file mode 100644 index 8ca4732..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_fence.c +++ /dev/null @@ -1,71 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_fence.c - * \brief ??? - */ -#include "mpidi_onesided.h" - - -int -MPID_Win_fence(int assert, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - static char FCNAME[] = "MPID_Win_fence"; - - if(win->mpid.sync.origin_epoch_type != win->mpid.sync.target_epoch_type){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - if (!(assert & MPI_MODE_NOPRECEDE) && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_FENCE && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_REFENCE && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_NONE) { - /* --BEGIN ERROR HANDLING-- */ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - /* --END ERROR HANDLING-- */ - } - - - struct MPIDI_Win_sync* sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - - if(assert & MPI_MODE_NOSUCCEED) - { - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_NONE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_NONE; - } - else{ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_REFENCE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_REFENCE; - } - - if (!(assert & MPI_MODE_NOPRECEDE)) - { - mpi_errno = MPIR_Barrier_impl(win->comm_ptr, &errflag); - } - - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_fetch_and_op.c b/src/mpid/pamid/src/onesided/mpid_win_fetch_and_op.c deleted file mode 100644 index c3e6583..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_fetch_and_op.c +++ /dev/null @@ -1,378 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_fetch_and_op.c.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -static pami_result_t -MPIDI_Fetch_and_op_using_pami_rmw(pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - int target_rank; - - MPID_assert(req != NULL); - target_rank = req->target.rank; - - pami_rmw_t params; - params=zero_rmw_parms; - params.dest=req->dest; - params.cookie=(void *)req; - params.done_fn=MPIDI_Win_DoneCB; - params.type = req->pami_datatype; - params.operation = req->pami_op; - params.local=req->user_buffer; /*result*/ - params.remote=req->win->mpid.info[target_rank].base_addr + req->offset + (size_t)req->origin.dt.map[0].DLOOP_VECTOR_BUF; - params.value=req->buffer; /* replaced value with origin */ - - rc = PAMI_Rmw(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return rc; -} - - -void -MPIDI_WinAtomicCB(pami_context_t context, - void * cookie, - const void * _hdr, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPIDI_AtomicHeader_t *ahdr = (MPIDI_AtomicHeader_t *) _hdr; - MPID_assert (ahdr != NULL); - MPID_assert (sizeof(MPIDI_AtomicHeader_t) == size); - MPIDI_AtomicHeader_t ack_hdr = *ahdr; - - void *dest_addr = ahdr->remote_addr; - int len; - len = MPIDU_Datatype_get_basic_size (ahdr->datatype); - - if (ahdr->atomic_type == MPIDI_WIN_REQUEST_COMPARE_AND_SWAP) { - - //overwrite value with result in ack_hdr - MPIR_Memcpy(ack_hdr.buf, dest_addr, len); - - if (MPIR_Compare_equal (&ahdr->test, dest_addr, ahdr->datatype)) - MPIR_Memcpy(dest_addr, ahdr->buf, len); - } - else if (ahdr->atomic_type == MPIDI_WIN_REQUEST_FETCH_AND_OP) { - //overwrite value with result - MPIR_Memcpy(ack_hdr.buf, dest_addr, len); - - MPI_User_function *uop; - int one = 1; - uop = MPIR_OP_HDL_TO_FN(ahdr->op); - - if (ahdr->op == MPI_REPLACE) - MPIR_Memcpy(dest_addr, ahdr->buf, len); - else if (ahdr->op == MPI_NO_OP); - else - (*uop) ((void *)ahdr->buf, dest_addr, &one, &ahdr->datatype); - } - else - MPID_abort(); - - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_WinAtomicAck, - .dest = sender, - .header = { - .iov_base = &ack_hdr, - .iov_len = sizeof(MPIDI_AtomicHeader_t), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - .hints = {0}, - }; - - pami_result_t rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); -} - -void -MPIDI_WinAtomicAckCB(pami_context_t context, - void * cookie, - const void * _hdr, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - int len; - MPIDI_AtomicHeader_t *ahdr = (MPIDI_AtomicHeader_t *) _hdr; - //We have a valid result addr - if (ahdr->result_addr != NULL) { - len = MPIDU_Datatype_get_basic_size (ahdr->datatype); - MPIR_Memcpy(ahdr->result_addr, ahdr->buf, len); - } - - MPIDI_Win_DoneCB(context, ahdr->request_addr, PAMI_SUCCESS); -} - - -pami_result_t -MPIDI_Atomic (pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - MPIDI_AtomicHeader_t atomic_hdr; - int len; - - len = MPIDU_Datatype_get_basic_size (req->origin.datatype); - assert(len <= MAX_ATOMIC_TYPE_SIZE); - if (req->buffer) - MPIR_Memcpy(atomic_hdr.buf, req->buffer, len); - if (req->type == MPIDI_WIN_REQUEST_COMPARE_AND_SWAP) - MPIR_Memcpy(atomic_hdr.test, req->compare_buffer, len); - - atomic_hdr.result_addr = req->user_buffer; - atomic_hdr.remote_addr = req->win->mpid.info[req->target.rank].base_addr + req->offset; - atomic_hdr.request_addr = req; - atomic_hdr.datatype = req->origin.datatype; - atomic_hdr.atomic_type = req->type; - atomic_hdr.op = req->op; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - MPID_assert (req->origin.dt.num_contig == 1); - ++sync->started; - - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_WinAtomic, - .dest = req->dest, - .header = { - .iov_base = &atomic_hdr, - .iov_len = sizeof(MPIDI_AtomicHeader_t), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - .hints = {0}, - }; - - rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; -} - - -#define FUNCNAME MPIDI_Fetch_and_op -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_Fetch_and_op(const void *origin_addr, void *result_addr, - MPI_Datatype datatype, int target_rank, - MPI_Aint target_disp, MPI_Op op, MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - MPIDI_Win_request *req; - int good_for_rmw=0; - int count = 1; - int shm_locked = 0; - - if(win->mpid.sync.origin_epoch_type == win->mpid.sync.target_epoch_type && - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_FENCE; win->mpid.sync.target_epoch_type = MPID_EPOTYPE_FENCE; - } - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_NONE || - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - pami_type_t pami_type; - pami_atomic_t pami_op; - - if (target_rank == MPI_PROC_NULL) - return MPI_SUCCESS; - - MPI_Datatype basic_type = MPI_DATATYPE_NULL; - MPIDU_Datatype_get_basic_type(datatype, basic_type); - if ((datatype == MPI_FLOAT_INT) || - (datatype == MPI_DOUBLE_INT) || - (datatype == MPI_LONG_INT) || - (datatype == MPI_SHORT_INT) || - (datatype == MPI_LONG_DOUBLE_INT)) - { - MPID_assert(basic_type == MPI_DATATYPE_NULL); - basic_type = datatype; - } - MPID_assert(basic_type != MPI_DATATYPE_NULL); - - if(MPIDI_Datatype_is_pami_rmw_supported(basic_type, &pami_type, op, &pami_op) ) { -#ifndef __BGQ__ - good_for_rmw = 1; -#endif - } else { - if((op == MPI_NO_OP) && (origin_addr == NULL) && (win->create_flavor != MPI_WIN_FLAVOR_SHARED) ) { - /* essentially a MPI_Get to result buffer */ - MPID_Get(result_addr, 1, datatype, target_rank, - target_disp, 1, datatype, win); - return 0; - } - } - - req = (MPIDI_Win_request *) MPL_calloc0(1, MPIDI_Win_request); - req->win = win; - req->type = MPIDI_WIN_REQUEST_FETCH_AND_OP; - - req->offset = target_disp * win->mpid.info[target_rank].disp_unit; -#ifdef __BGQ__ - /* PAMI limitation as it doesnt permit VA of 0 to be passed into - * memregion create, so we must pass base_va of heap computed from - * an SPI call instead. So the target offset must be adjusted */ - if (req->win->create_flavor == MPI_WIN_FLAVOR_DYNAMIC) - req->offset -= (size_t)req->win->mpid.info[target_rank].base_addr; -#endif - - if (datatype == MPI_DOUBLE_INT) - { - MPIDI_Win_datatype_basic(count*2, - MPI_DOUBLE, - &req->origin.dt); - } - else if (datatype == MPI_LONG_DOUBLE_INT) - { - MPIDI_Win_datatype_basic(count*2, - MPI_LONG_DOUBLE, - &req->origin.dt); - } - else if (datatype == MPI_LONG_INT) - { - MPIDI_Win_datatype_basic(count*2, - MPI_LONG, - &req->origin.dt); - } - else if (datatype == MPI_SHORT_INT) - { - MPIDI_Win_datatype_basic(count*2, - MPI_INT, - &req->origin.dt); - } - else - { - MPIDI_Win_datatype_basic(count, - datatype, - &req->origin.dt); - } - - - if (req->origin.dt.size == 0) - { - MPL_free(req); - return MPI_SUCCESS; - } - - req->target.rank = target_rank; - - req->compare_buffer = NULL; - req->pami_op = pami_op; - req->op = op; - req->pami_datatype = pami_type; - /* MPI_Fetch_and_op only supports predefined datatype */ - req->buffer = (void *) ((uintptr_t) origin_addr + req->origin.dt.true_lb); - req->user_buffer = result_addr + req->origin.dt.true_lb; - - pami_result_t rc; - pami_task_t task = MPID_VCR_GET_LPID(win->comm_ptr->vcr, target_rank); - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_START && - !MPIDI_valid_group_rank(task, win->mpid.sync.sc.group)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &req->dest); - MPID_assert(rc == PAMI_SUCCESS); - - MPIDI_Win_datatype_map(&req->origin.dt); - win->mpid.sync.total += req->origin.dt.num_contig; - req->origin.datatype= basic_type; - - /* The pamid one-sided design requires context post in order to handle the - * case where the number of pending rma operation exceeds the - * 'PAMID_RMA_PENDING' threshold. When there are too many pending requests the - * work function remains on the context post queue (by returning PAMI_EAGAIN) - * so that the next time the context is advanced the work function will be - * invoked again. - * - * TODO - When context post is not required it would be better to attempt a - * direct context operation and then fail over to using context post if - * the rma pending threshold has been reached. This would result in - * better latency for one-sided operations. - */ - if(good_for_rmw) { - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Fetch_and_op_using_pami_rmw, req); - } else { - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Atomic, req); - - } - -fn_fail: - return mpi_errno; -} - - -int MPIDI_Datatype_is_pami_rmw_supported(MPI_Datatype datatype, pami_type_t *pami_type, MPI_Op op, pami_atomic_t *pami_op) -{ - int null=0; - MPI_Op null_op=0; - int rc = FALSE; - pami_data_function pami_data_fn; - - MPIDI_Datatype_to_pami(datatype, pami_type, op, &pami_data_fn, &null); - - if(*pami_type == PAMI_TYPE_SIGNED_INT || - *pami_type == PAMI_TYPE_UNSIGNED_INT || - *pami_type == PAMI_TYPE_SIGNED_LONG || - *pami_type == PAMI_TYPE_UNSIGNED_LONG || - *pami_type == PAMI_TYPE_SIGNED_LONG_LONG || - *pami_type == PAMI_TYPE_SIGNED_LONG_LONG) { - if(op == null_op) { - rc = TRUE; - } else if (op == MPI_SUM) { - *pami_op = PAMI_ATOMIC_FETCH_ADD; - rc = TRUE; - } else if (op == MPI_BOR) { - *pami_op = PAMI_ATOMIC_FETCH_OR; - rc = TRUE; - } else if (op == MPI_BAND) { - *pami_op = PAMI_ATOMIC_FETCH_AND; - rc = TRUE; - } else if (op == MPI_BXOR) { - *pami_op = PAMI_ATOMIC_FETCH_XOR; - rc = TRUE; - } else if (op == MPI_REPLACE) { - *pami_op = PAMI_ATOMIC_FETCH_SET; - rc = TRUE; - } else if (op == MPI_NO_OP) { - *pami_op = PAMI_ATOMIC_FETCH; - rc = TRUE; - } - } - return rc; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_flush.c b/src/mpid/pamid/src/onesided/mpid_win_flush.c deleted file mode 100644 index 370bbd5..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_flush.c +++ /dev/null @@ -1,180 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_flush.c - * \brief returns a new info object containing the hints of the window - * associated with win. - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_flush function - * - * The funcion can be called only within passive target epochs such as - * MPI_Win_lock, MPI_Win_unlock, MPI_Win_lock_all and MPI_Win_unlock_all. - * - * The function completes all outstanding RMA operations initialized by - * the calling process to a specified target rank on the given window. - * The operations are completed both at the origin and the target. - * - * \param[in] rank rank of target window - * \param[in] win window object - * \return MPI_SUCCESS, MPI_ERR_OTHER - */ - - -int -MPID_Win_flush(int rank, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIDI_Win_sync* sync; - static char FCNAME[] = "MPID_Win_flush"; - - if((win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK) && - (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_DO_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - - return mpi_errno; -} -/** - * \brief MPI-PAMI glue for MPI_Win_flush_all function - * - * The funcion can be called only within passive target epochs such as - * MPI_Win_lock, MPI_Win_unlock, MPI_Win_lock_all and MPI_Win_unlock_all. - * - * All RMA opertions issued by the calling process to any target on the - * given window prior to this call and in the given window will have - * completed both at the origin and the target when the call returns. - * - * \param[in] win window object - * \return MPI_SUCCESS, MPI_ERR_OTHER - */ - - -int -MPID_Win_flush_all(MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIDI_Win_sync* sync; - static char FCNAME[] = "MPID_Win_flush_all"; - - if((win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK) && - (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - return mpi_errno; - -} - -/** - * \brief MPI-PAMI glue for MPI_Win_flush_local function - * - * The funcion can be called only within passive target epochs such as - * MPI_Win_lock, MPI_Win_unlock, MPI_Win_lock_all and MPI_Win_unlock_all. - * - * Locally completes at the origin all outstanding RMA operations initiated - * the cng process to the target rank on the given window. The user may - * reuse any buffers after this routine returns. - * - * It has been determined that the routine uses only counters for each window - * and not for each rank because the overhead of tracking each rank could be great - * if a window group contains a large number of ranks. - * - * \param[in] rank rank of target window - * \param[in] win window object - * \return MPI_SUCCESS, MPI_ERR_OTHER - */ - -int -MPID_Win_flush_local(int rank, MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIDI_Win_sync* sync; - static char FCNAME[] = "MPID_Win_flush_local"; - - if((win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK) && - (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - - return mpi_errno; -} - -/** - * \brief MPI-PAMI glue for MPI_Win_flush_local_all function - * - * The funcion can be called only within passive target epochs such as - * MPI_Win_lock, MPI_Win_unlock, MPI_Win_lock_all and MPI_Win_unlock_all. - * - * All RMA operations issued to any target prior to this call in this window - * will have copleted at the origin when this function returns. - * - * \param[in] win window object - * \return MPI_SUCCESS, MPI_ERR_OTHER - */ - -int -MPID_Win_flush_local_all(MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIDI_Win_sync* sync; - static char FCNAME[] = "MPID_Win_flush"; - - if((win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK) && - (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - - return mpi_errno; -} - - - - - diff --git a/src/mpid/pamid/src/onesided/mpid_win_free.c b/src/mpid/pamid/src/onesided/mpid_win_free.c deleted file mode 100644 index 2533706..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_free.c +++ /dev/null @@ -1,132 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_free.c - * \brief ??? - */ -#include "mpidi_onesided.h" -#include <sys/shm.h> -#include <sys/ipc.h> -#include <sys/stat.h> -#ifdef USE_MMAP_SHM -#include <sys/mman.h> -#endif - - -int MPIDI_SHM_Win_free(MPIR_Win **win_ptr) -{ - static char FCNAME[] = "MPID_SHM_Win_free"; - int rc; - int mpi_errno = MPI_SUCCESS; - - /* Free shared memory region */ - /* free shm_base_addrs that's only used for shared memory windows */ - if ((*win_ptr)->mpid.shm->allocated) { - OPA_fetch_and_add_int((OPA_int_t *) &((*win_ptr)->mpid.shm->ctrl->shm_count),-1); - while((*win_ptr)->mpid.shm->ctrl->shm_count !=0) MPIDI_QUICKSLEEP; - if ((*win_ptr)->comm_ptr->rank == 0) { - MPIDI_SHM_MUTEX_DESTROY(*win_ptr); - } -#ifdef USE_SYSV_SHM - mpi_errno = shmdt((*win_ptr)->mpid.shm->base_addr); - if ((*win_ptr)->comm_ptr->rank == 0) { - rc=shmctl((*win_ptr)->mpid.shm->shm_id,IPC_RMID,NULL); - MPIU_ERR_CHKANDJUMP((rc == -1), errno,MPI_ERR_RMA_SHARED, "**shmctl"); - } -#elif USE_MMAP_SHM - munmap ((*win_ptr)->mpid.shm->base_addr, (*win_ptr)->mpid.shm->segment_len); - if (0 == (*win_ptr)->comm_ptr->rank) shm_unlink ((*win_ptr)->mpid.shm->shm_key); -#else - MPID_Abort(NULL, MPI_ERR_RMA_SHARED, -1, "MPI_Win_free error"); -#endif - } else {/* one task on a node */ - MPL_free((*win_ptr)->mpid.shm->base_addr); - } - MPL_free((*win_ptr)->mpid.shm); - (*win_ptr)->mpid.shm = NULL; - - fn_fail: - return mpi_errno; -} - -/** - * \brief MPI-PAMI glue for MPI_Win_free function - * - * Release all references and free memory associated with window. - * - * \param[in,out] win Window - * \return MPI_SUCCESS or error returned from MPI_Barrier. - */ -#undef FUNCNAME -#define FUNCNAME MPID_Win_free -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Win_free(MPIR_Win **win_ptr) -{ - int mpi_errno = MPI_SUCCESS; - - MPIR_Win *win = *win_ptr; - size_t rank = win->comm_ptr->rank; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - - if(win->mpid.sync.origin_epoch_type != win->mpid.sync.target_epoch_type || - (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_NONE && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_REFENCE)){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, return mpi_errno, "**rmasync"); - } - - mpi_errno = MPIR_Barrier_impl(win->comm_ptr, &errflag); - MPIU_ERR_CHKANDJUMP(errflag, mpi_errno, MPI_ERR_OTHER, "**mpi_bcast"); - - if (win->create_flavor == MPI_WIN_FLAVOR_SHARED) - mpi_errno=MPIDI_SHM_Win_free(win_ptr); - - - - if (win->create_flavor == MPI_WIN_FLAVOR_ALLOCATE) - MPL_free(win->base); - - struct MPIDI_Win_info *winfo = &win->mpid.info[rank]; -#ifdef USE_PAMI_RDMA - if (win->size != 0) - { - pami_result_t rc; - rc = PAMI_Memregion_destroy(MPIDI_Context[0], &winfo->memregion); - MPID_assert(rc == PAMI_SUCCESS); - } -#else - if ( (!MPIDI_Process.mp_s_use_pami_get) && (win->size != 0) && (winfo->memregion_used) ) - { - pami_result_t rc; - rc = PAMI_Memregion_destroy(MPIDI_Context[0], &winfo->memregion); - MPID_assert(rc == PAMI_SUCCESS); - } -#endif - - MPL_free(win->mpid.info); - if (win->mpid.work.msgQ) - MPL_free(win->mpid.work.msgQ); - - MPIR_Comm_release(win->comm_ptr, 0); - - MPIR_Handle_obj_free(&MPIR_Win_mem, win); - -fn_fail: - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_get.c b/src/mpid/pamid/src/onesided/mpid_win_get.c deleted file mode 100644 index f9cd4b3..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_get.c +++ /dev/null @@ -1,434 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_get.c - * \brief ??? - */ -#include "mpidi_onesided.h" -#include "mpidi_util.h" - -static inline int -MPIDI_Get_use_pami_rget(pami_context_t context, MPIDI_Win_request * req) -__attribute__((__always_inline__)); -#ifdef RDMA_FAILOVER -static inline int -MPIDI_Get_use_pami_get(pami_context_t context, MPIDI_Win_request * req) -__attribute__((__always_inline__)); -#endif - -static pami_result_t -MPIDI_Get(pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - -#ifdef USE_PAMI_RDMA - rc = MPIDI_Get_use_pami_rget(context,req); -#else - if( (req->origin.memregion_used) && - (req->win->mpid.info[req->target.rank].memregion_used) ) - { - rc = MPIDI_Get_use_pami_rget(context,req); - } else { - rc = MPIDI_Get_use_pami_get(context,req); - } -#endif - if(rc == PAMI_EAGAIN) - return rc; - - return PAMI_SUCCESS; -} - - -static inline int -MPIDI_Get_use_pami_rget(pami_context_t context, MPIDI_Win_request * req) -{ - int use_typed_rdma = 0; - - if (!req->target.dt.contig || !req->origin.dt.contig) { - use_typed_rdma = 0; - if (MPIDI_Process.typed_onesided == 1) - use_typed_rdma = 1; - } - - if (use_typed_rdma) { - pami_result_t rc; - pami_rget_typed_t params; - /* params need to zero out to avoid passing garbage to PAMI */ - params=zero_rget_typed_parms; - - params.rma.dest=req->dest; - params.rma.hints.buffer_registered = PAMI_HINT_ENABLE; - params.rma.hints.use_rdma = PAMI_HINT_ENABLE; - params.rma.bytes = req->target.dt.size; - params.rma.cookie = req; - params.rma.done_fn = MPIDI_Win_DoneCB; - params.rdma.local.mr=&req->origin.memregion; - params.rdma.remote.mr=&req->win->mpid.info[req->target.rank].memregion; - params.rdma.remote.offset= req->offset; - params.rdma.local.offset = req->state.local_offset; - - params.type.local = *(pami_type_t *)(req->origin.dt.pointer->device_datatype); - params.type.remote = *(pami_type_t *)(req->target.dt.pointer->device_datatype); - - - rc = PAMI_Rget_typed(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - } - else { - pami_result_t rc; - pami_rget_simple_t params; - - params=zero_rget_parms; - - params.rma.dest=req->dest; - params.rma.hints.buffer_registered = PAMI_HINT_ENABLE; - params.rma.hints.use_rdma = PAMI_HINT_ENABLE; - params.rma.bytes = 0; - params.rma.cookie = req; - params.rma.done_fn = MPIDI_Win_DoneCB; - params.rdma.local.mr=&req->origin.memregion; - params.rdma.remote.mr=&req->win->mpid.info[req->target.rank].memregion; - params.rdma.remote.offset= req->offset; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - TRACE_ERR("Start index=%u/%d l-addr=%p r-base=%p r-offset=%zu (sync->started=%u sync->complete=%u)\n", - req->state.index, req->target.dt.num_contig, req->buffer, req->win->mpid.info[req->target.rank].base_addr, req->offset, sync->started, sync->complete); - while (req->state.index < req->target.dt.num_contig) { - if (sync->started > sync->complete + MPIDI_Process.rma_pending) - { - TRACE_ERR("Bailing out; index=%u/%d sync->started=%u sync->complete=%u\n", - req->state.index, req->target.dt.num_contig, sync->started, sync->complete); - return PAMI_EAGAIN; - } - ++sync->started; - - params.rma.bytes = req->target.dt.map[req->state.index].DLOOP_VECTOR_LEN; - params.rdma.remote.offset = req->offset + (size_t)req->target.dt.map[req->state.index].DLOOP_VECTOR_BUF; - params.rdma.local.offset = req->state.local_offset; - -#ifdef TRACE_ON - unsigned* buf = (unsigned*)(req->buffer + params.rdma.local.offset); -#endif - TRACE_ERR(" Sub index=%u bytes=%zu l-offset=%zu r-offset=%zu buf=%p *(int*)buf=0x%08x\n", req->state.index, params.rma.bytes, params.rdma.local.offset, params.rdma.remote.offset, buf, *buf); - - /** sync->total will be updated with every RMA and the complete - will not change till that RMA has completed. In the meanwhile - the rest of the RMAs will have memory leaks */ - if (req->target.dt.num_contig - req->state.index == 1) { - rc = PAMI_Rget(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; - } else { - rc = PAMI_Rget(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - req->state.local_offset += params.rma.bytes; - ++req->state.index; - } - } - } - return PAMI_SUCCESS; -} - - -#ifdef RDMA_FAILOVER -static inline int -MPIDI_Get_use_pami_get(pami_context_t context, MPIDI_Win_request * req) -{ - pami_result_t rc; - pami_get_simple_t params; - - params=zero_get_parms; - - params.rma.dest=req->dest; - params.rma.hints.use_rdma = PAMI_HINT_DEFAULT; -#ifndef OUT_OF_ORDER_HANDLING - params.rma.hints.no_long_header= 1, -#endif - params.rma.bytes = 0; - params.rma.cookie = req; - params.rma.done_fn = MPIDI_Win_DoneCB; - params.addr.local=req->buffer; - params.addr.remote= req->win->mpid.info[req->target.rank].base_addr; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - TRACE_ERR("Start index=%u/%d l-addr=%p r-base=%p r-offset=%zu (sync->started=%u sync->complete=%u)\n", - req->state.index, req->target.dt.num_contig, req->buffer, req->win->mpid.info[req->target.rank].base_addr, req->offset, sync->started, sync->complete); - while (req->state.index < req->target.dt.num_contig) { - if (sync->started > sync->complete + MPIDI_Process.rma_pending) - { - TRACE_ERR("Bailing out; index=%u/%d sync->started=%u sync->complete=%u\n", - req->state.index, req->target.dt.num_contig, sync->started, sync->complete); - return PAMI_EAGAIN; - } - ++sync->started; - - - params.rma.bytes = req->target.dt.map[req->state.index].DLOOP_VECTOR_LEN; - params.addr.local = req->buffer+req->state.local_offset; - params.addr.remote = req->win->mpid.info[req->target.rank].base_addr+ req->offset + (size_t)req->target.dt.map[req->state.index].DLOOP_VECTOR_BUF; - -#ifdef TRACE_ON - unsigned* buf = (unsigned*)(req->buffer + params.rdma.local.offset); -#endif - TRACE_ERR(" Sub index=%u bytes=%zu l-offset=%zu r-offset=%zu buf=%p *(int*)buf=0x%08x\n", - req->state.index, params.rma.bytes, params.rdma.local.offset, params.rdma.remote.offset, buf, *buf); - - /** sync->total will be updated with every RMA and the complete - will not change till that RMA has completed. In the meanwhile - the rest of the RMAs will have memory leaks */ - if (req->target.dt.num_contig - req->state.index == 1) { - rc = PAMI_Get(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; - } else { - rc = PAMI_Get(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - req->state.local_offset += params.rma.bytes; - ++req->state.index; - } - } - return PAMI_SUCCESS; -} -#endif - - -/** - * \brief MPI-PAMI glue for MPI_GET function - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Get -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Get(void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPIR_Win *win) -{ - - int mpi_errno = MPI_SUCCESS; - int shm_locked=0; - void *target_addr; - MPIDI_Win_request *req = MPL_calloc0(1, MPIDI_Win_request); - req->win = win; - if(win->mpid.request_based != 1) - req->type = MPIDI_WIN_REQUEST_GET; - else { - req->req_handle = win->mpid.rreq; - req->type = MPIDI_WIN_REQUEST_RGET; - req->req_handle->mpid.win_req = req; - } - - if(win->mpid.sync.origin_epoch_type == win->mpid.sync.target_epoch_type && - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_FENCE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_FENCE; - } - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_NONE || - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - req->offset = target_disp * win->mpid.info[target_rank].disp_unit; -#ifdef __BGQ__ - /* PAMI limitation as it doesnt permit VA of 0 to be passed into - * memregion create, so we must pass base_va of heap computed from - * an SPI call instead. So the target offset must be adjusted */ - if (req->win->create_flavor == MPI_WIN_FLAVOR_DYNAMIC) - req->offset -= (size_t)req->win->mpid.info[target_rank].base_addr; -#endif - - MPIDI_Win_datatype_basic(origin_count, - origin_datatype, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count, - target_datatype, - &req->target.dt); - #ifndef MPIDI_NO_ASSERT - MPID_assert(req->origin.dt.size == req->target.dt.size); - #else - MPIU_ERR_CHKANDJUMP((req->origin.dt.size != req->target.dt.size), mpi_errno, MPI_ERR_SIZE, "**rmasize"); - #endif - - if ( (req->origin.dt.size == 0) || - (target_rank == MPI_PROC_NULL)) - { - if(req->req_handle) - MPIR_cc_set(req->req_handle->cc_ptr, 0); - else - MPL_free(req); - return MPI_SUCCESS; - } - - /* If the get is a local operation,or shared window do it here */ - if ((target_rank == win->comm_ptr->rank) || (win->create_flavor == MPI_WIN_FLAVOR_SHARED)) - { - if (target_rank == win->comm_ptr->rank) - target_addr = win->base + req->offset; - else - target_addr = win->mpid.info[target_rank].base_addr + req->offset; - - - /* The operation is not complete until the local copy is performed */ - mpi_errno = MPIR_Localcopy(target_addr, - target_count, - target_datatype, - origin_addr, - origin_count, - origin_datatype); - - /* The instant this completion counter is set to zero another thread - * may notice the change and begin freeing request resources. The - * thread executing the code in this function must not touch any - * portion of the request structure after decrementing the completion - * counter. - * - * See MPID_Request_free_inline() - */ - if(req->req_handle) - MPIR_cc_set(req->req_handle->cc_ptr, 0); - else - MPL_free(req); - return mpi_errno; - } - req->target.rank = target_rank; - - - /* Only pack the origin data if the origin is non-contiguous and we are using the simple PAMI_Rget. - * If we are using the typed PAMI_Rget_typed use the origin address as is, if we are using the simple - * PAMI_Rget with contiguous data use the origin address with the lower-bound adjustment. - */ - if (req->origin.dt.contig || (!req->origin.dt.contig && (MPIDI_Process.typed_onesided == 1))) - { - req->buffer_free = 0; - if ((req->origin.dt.contig && req->target.dt.contig && (MPIDI_Process.typed_onesided == 1)) || (!(MPIDI_Process.typed_onesided == 1))) // use simple rput - req->buffer = (void *) ((uintptr_t) origin_addr + req->origin.dt.true_lb); - else - req->buffer = (void *) ((uintptr_t) origin_addr); - } - else - { - req->buffer_free = 1; - req->buffer = MPL_malloc(req->origin.dt.size); - MPID_assert(req->buffer != NULL); - - MPIDU_Datatype_add_ref(req->origin.dt.pointer); - req->origin.addr = origin_addr; - req->origin.count = origin_count; - req->origin.datatype = origin_datatype; - req->origin.completed = 0; - - - - - } - - - pami_result_t rc; - pami_task_t task = MPID_VCR_GET_LPID(win->comm_ptr->vcr, target_rank); - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_START && - !MPIDI_valid_group_rank(task, win->mpid.sync.sc.group)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &req->dest); - MPID_assert(rc == PAMI_SUCCESS); - -#ifdef USE_PAMI_RDMA - size_t length_out; - rc = PAMI_Memregion_create(MPIDI_Context[0], - req->buffer, - req->origin.dt.size, - &length_out, - &req->origin.memregion); - MPID_assert(rc == PAMI_SUCCESS); - MPID_assert(req->origin.dt.size == length_out); -#else - if(!MPIDI_Process.mp_s_use_pami_get) - { - size_t length_out; - rc = PAMI_Memregion_create(MPIDI_Context[0], - req->buffer, - req->origin.dt.size, - &length_out, - &req->origin.memregion); - if(rc == PAMI_SUCCESS) - { - req->origin.memregion_used = 1; - MPID_assert(req->origin.dt.size == length_out); - } - } -#endif - - - if ((!req->target.dt.contig || !req->origin.dt.contig) && (MPIDI_Process.typed_onesided == 1)) - /* If the datatype is non-contiguous and the PAMID typed_onesided optimization - * is enabled then we will be using the typed interface and will only make 1 call. - */ - win->mpid.sync.total = 1; - else { - MPIDI_Win_datatype_map(&req->target.dt); - win->mpid.sync.total += req->target.dt.num_contig; - } - - if ((MPIDI_Process.typed_onesided == 1) && (!req->target.dt.contig || !req->origin.dt.contig)) { - /* We will use the PAMI_Rget_typed call so we need to make sure any MPI_Type_free before the context - * executes the get does not free the MPIDU_Datatype, which would also free the associated PAMI datatype which - * is still needed for communication -- decrement the ref in the callback to allow the MPIR_Datatype - * to be freed once the PAMI communication has completed. - */ - MPIDU_Datatype_add_ref(req->origin.dt.pointer); - MPIDU_Datatype_add_ref(req->target.dt.pointer); - } - /* The pamid one-sided design requires context post in order to handle the - * case where the number of pending rma operation exceeds the - * 'PAMID_RMA_PENDING' threshold. When there are too many pending requests the - * work function remains on the context post queue (by returning PAMI_EAGAIN) - * so that the next time the context is advanced the work function will be - * invoked again. - * - * TODO - When context post is not required it would be better to attempt a - * direct context operation and then fail over to using context post if - * the rma pending threshold has been reached. This would result in - * better latency for one-sided operations. - */ - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Get, req); - -fn_fail: - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_get_accumulate.c b/src/mpid/pamid/src/onesided/mpid_win_get_accumulate.c deleted file mode 100644 index d1a6ae0..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_get_accumulate.c +++ /dev/null @@ -1,586 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_accumulate.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -static void -MPIDI_Fetch_data_op(const void * origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - void * result_addr, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPI_Op op, - MPIR_Win *win) -{ - static char FCNAME[] = "MPIDI_Fetch_data_op"; - int shm_locked=0; - MPI_User_function *uop; - void *base, *dest_addr; - int disp_unit; - int len, one; - - base = win->mpid.info[target_rank].base_addr; - disp_unit = win->mpid.info[target_rank].disp_unit; - dest_addr = (char *) base + disp_unit * target_disp; - - MPIDU_Datatype_get_size_macro(origin_datatype, len); - MPIR_Memcpy(result_addr, dest_addr, len); - if (op != MPI_NO_OP) { - uop = MPIR_OP_HDL_TO_FN(op); - one = 1; - (*uop)((void *) origin_addr, dest_addr, &one, &origin_datatype); - } - - fn_fail: return; -} - - -static void -MPIDI_Win_GetAccSendAckDoneCB(pami_context_t context, - void * _info, - pami_result_t result) -{ - MPIDI_Win_GetAccMsgInfo *msginfo = (MPIDI_Win_GetAccMsgInfo *) _info; - MPL_free(msginfo->tptr); - MPL_free(msginfo); -} - -static void -MPIDI_Win_GetAccumSendAck(pami_context_t context, - void * _info, - pami_result_t result) -{ - static char FCNAME[] = "MPID_Win_GetAccumSendAck"; - MPIDI_Win_GetAccMsgInfo *msginfo = (MPIDI_Win_GetAccMsgInfo *) _info; - pami_result_t rc = PAMI_SUCCESS; - - //Copy from msginfo->addr to a contiguous buffer - char *buffer = NULL; - - buffer = MPL_malloc(msginfo->size); - MPID_assert(buffer != NULL); - - if (msginfo->num_contig == 1) - memcpy(buffer, msginfo->addr, msginfo->size); - else - { - int mpi_errno = 0; - mpi_errno = MPIR_Localcopy(msginfo->addr, - msginfo->count, - msginfo->type, - buffer, - msginfo->size, - MPI_CHAR); - MPID_assert(mpi_errno == MPI_SUCCESS); - } - //Schedule sends to source to result buffer and trigger completion - //callback there - pami_send_t params = { - .send = { - .header = { - .iov_base = msginfo, - .iov_len = sizeof(MPIDI_Win_GetAccMsgInfo), - }, - .dispatch = MPIDI_Protocols_WinGetAccumAck, - .dest = msginfo->src_endpoint, - }, - .events = { - .cookie = msginfo, - .local_fn = MPIDI_Win_GetAccSendAckDoneCB, //cleanup buffer - }, - }; - - msginfo->tptr = buffer; - - params.send.data.iov_len = msginfo->size; - params.send.data.iov_base = buffer; - - rc = PAMI_Send(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - fn_fail: return; -} - -void -MPIDI_WinGetAccumCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv != NULL); - MPID_assert(sndbuf == NULL); - MPID_assert(msginfo_size == sizeof(MPIDI_Win_GetAccMsgInfo)); - MPID_assert(_msginfo != NULL); - MPIDI_Win_GetAccMsgInfo * msginfo = (MPIDI_Win_GetAccMsgInfo *) - MPL_malloc(sizeof(MPIDI_Win_GetAccMsgInfo)); - - *msginfo = *(MPIDI_Win_GetAccMsgInfo *)_msginfo; - msginfo->src_endpoint = sender; - - int null=0; - pami_type_t pami_type; - pami_data_function pami_op; - MPI_Op op = msginfo->op; - - MPIDI_Datatype_to_pami(msginfo->type, &pami_type, op, &pami_op, &null); - - recv->addr = msginfo->addr; - recv->type = pami_type; - recv->offset = 0; - recv->data_fn = pami_op; - recv->data_cookie = NULL; - recv->local_fn = NULL; - recv->cookie = NULL; - - if (msginfo->counter == 0) - //We will now allocate a tempbuf, copy local contents and start a - //send - MPIDI_Win_GetAccumSendAck (context, msginfo, PAMI_SUCCESS); - else - MPL_free(msginfo); -} - -static void -MPIDI_Win_GetAccDoneCB(pami_context_t context, - void * cookie, - pami_result_t result) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)cookie; - ++req->win->mpid.sync.complete; - ++req->origin.completed; - - if (req->origin.completed == req->target.dt.num_contig + 1) - { - MPIR_Request * req_handle = req->req_handle; - - if (req->buffer_free) { - MPL_free(req->buffer); - req->buffer_free = 0; - } - if (req->accum_headers) - MPL_free(req->accum_headers); - - MPIDI_Win_datatype_unmap(&req->target.dt); - MPIDI_Win_datatype_unmap(&req->result.dt); - - if( req->type != MPIDI_WIN_REQUEST_RGET_ACCUMULATE ) { - if (req_handle) { - req_handle->mpid.win_req = NULL; - } - MPL_free(req); - } - /* The instant this completion counter is set to zero another thread - * may notice the change and begin freeing request resources. The - * thread executing the code in this function must not touch any - * portion of the request structure after decrementing the completion - * counter. - * - * See MPID_Request_free_inline() - */ - if(req_handle) - MPIR_cc_set(req_handle->cc_ptr, 0); - } - MPIDI_Progress_signal(); -} - -void -MPIDI_Win_GetAccAckDoneCB(pami_context_t context, - void * _msginfo, - pami_result_t result) -{ - MPIDI_Win_GetAccMsgInfo * msginfo =(MPIDI_Win_GetAccMsgInfo *)_msginfo; - MPIDI_Win_request *req = (MPIDI_Win_request *) msginfo->request; - - if (req->result_num_contig > 1) { - MPIR_Localcopy(req->result.addr, - req->result.count, - req->result.datatype, - msginfo->tptr, - msginfo->size, - MPI_CHAR); - MPL_free(msginfo->tptr); - } - MPL_free(msginfo); - - MPIDI_Win_GetAccDoneCB(context, req, result); -} - - -void -MPIDI_WinGetAccumAckCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv != NULL); - MPID_assert(sndbuf == NULL); - MPID_assert(_msginfo != NULL); - MPIDI_Win_GetAccMsgInfo * msginfo =MPL_malloc(sizeof(MPIDI_Win_GetAccMsgInfo)); - *msginfo = *(const MPIDI_Win_GetAccMsgInfo *)_msginfo; - MPIDI_Win_request *req = (MPIDI_Win_request *) msginfo->request; - - msginfo->tptr = NULL; - recv->addr = req->result.addr; - if (req->result_num_contig > 1) - recv->addr = msginfo->tptr = MPL_malloc(msginfo->size); - - recv->type = PAMI_TYPE_BYTE; - recv->offset = 0; - recv->data_fn = PAMI_DATA_COPY; - recv->data_cookie = NULL; - recv->local_fn = MPIDI_Win_GetAccAckDoneCB; - recv->cookie = msginfo; -} - -static pami_result_t -MPIDI_Get_accumulate(pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - - pami_send_t params = { - .send = { - .header = { - .iov_len = sizeof(MPIDI_Win_GetAccMsgInfo), - }, - .dispatch = MPIDI_Protocols_WinGetAccum, - .dest = req->dest, - }, - .events = { - .cookie = req, - .remote_fn = MPIDI_Win_GetAccDoneCB, //When all accumulates have - //completed remotely - //complete accumulate - }, - }; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - TRACE_ERR("Start index=%u/%d l-addr=%p r-base=%p r-offset=%zu (sync->started=%u sync->complete=%u)\n", - req->state.index, req->target.dt.num_contig, req->buffer, req->win->mpid.info[req->target.rank].base_addr, req->offset, sync->started, sync->complete); - while (req->state.index < req->target.dt.num_contig) { - if (sync->started > sync->complete + MPIDI_Process.rma_pending) - { - TRACE_ERR("Bailing out; index=%u/%d sync->started=%u sync->complete=%u\n", - req->state.index, req->target.dt.num_contig, sync->started, sync->complete); - return PAMI_EAGAIN; - } - ++sync->started; - - params.send.header.iov_base = &(((MPIDI_Win_GetAccMsgInfo *)req->accum_headers)[req->state.index]); - params.send.data.iov_len = req->target.dt.map[req->state.index].DLOOP_VECTOR_LEN; - params.send.data.iov_base = req->buffer + req->state.local_offset; - - if (req->target.dt.num_contig - req->state.index == 1) { - rc = PAMI_Send(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; - } else { - rc = PAMI_Send(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - req->state.local_offset += params.send.data.iov_len; - ++req->state.index; - } - } - return PAMI_SUCCESS; -} - - -/** - * \brief MPI-PAMI glue for MPI_GET_ACCUMULATE function - * - * According to the MPI Specification: - * - * Each datatype argument must be a predefined datatype or - * a derived datatype, where all basic components are of the - * same predefined datatype. Both datatype arguments must be - * constructed from the same predefined datatype. - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] op Operand to perform - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Get_accumulate -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Get_accumulate(const void * origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - void * result_addr, - int result_count, - MPI_Datatype result_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPI_Op op, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - - if (op == MPI_NO_OP) {//we just need to fetch data - if (win->create_flavor == MPI_WIN_FLAVOR_SHARED) { - win->mpid.sync.total++; - MPIDI_Fetch_data_op(origin_addr, origin_count, origin_datatype, - result_addr, target_rank, target_disp, - target_count, target_datatype, op, win); - ++win->mpid.sync.complete; - } else { - mpi_errno = MPID_Get(result_addr, - result_count, - result_datatype, - target_rank, - target_disp, - target_count, - target_datatype, - win); - } - return mpi_errno; - } - - MPIDI_Win_request *req; - req = MPL_calloc0(1, MPIDI_Win_request); - req->win = win; - if(win->mpid.request_based != 1) - req->type = MPIDI_WIN_REQUEST_GET_ACCUMULATE; - else { - req->req_handle = win->mpid.rreq; - req->type = MPIDI_WIN_REQUEST_RGET_ACCUMULATE; - req->req_handle->mpid.win_req = req; - } - req->offset = target_disp*win->mpid.info[target_rank].disp_unit; -#ifdef __BGQ__ - /* PAMI limitation as it doesnt permit VA of 0 to be passed into - * memregion create, so we must pass base_va of heap computed from - * an SPI call instead. So the target offset must be adjusted */ - if (req->win->create_flavor == MPI_WIN_FLAVOR_DYNAMIC) - req->offset -= (size_t)req->win->mpid.info[target_rank].base_addr; -#endif - - if(win->mpid.sync.origin_epoch_type == win->mpid.sync.target_epoch_type && - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_FENCE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_FENCE; - } - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_NONE || - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - req->offset = target_disp * win->mpid.info[target_rank].disp_unit; - - if (origin_datatype == MPI_DOUBLE_INT) { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_DOUBLE, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_DOUBLE, - &req->target.dt); - } - else if (origin_datatype == MPI_LONG_DOUBLE_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_LONG_DOUBLE, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_LONG_DOUBLE, - &req->target.dt); - } - else if (origin_datatype == MPI_LONG_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_LONG, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_LONG, - &req->target.dt); - } - else if (origin_datatype == MPI_SHORT_INT) - { - MPIDI_Win_datatype_basic(origin_count*2, - MPI_INT, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count*2, - MPI_INT, - &req->target.dt); - } - else - { - MPIDI_Win_datatype_basic(origin_count, - origin_datatype, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count, - target_datatype, - &req->target.dt); - } - - MPID_assert(req->origin.dt.size == req->target.dt.size); - - if ( (req->origin.dt.size == 0) || - (target_rank == MPI_PROC_NULL)) - { - if(req->req_handle) - MPIR_cc_set(req->req_handle->cc_ptr, 0); - else - MPL_free(req); - return MPI_SUCCESS; - } - - req->target.rank = target_rank; - - - if (req->origin.dt.contig) - { - req->buffer_free = 0; - req->buffer = (char*)origin_addr + req->origin.dt.true_lb; - } - else - { - req->buffer_free = 1; - req->buffer = MPL_malloc(req->origin.dt.size); - MPID_assert(req->buffer != NULL); - - int mpi_errno = 0; - mpi_errno = MPIR_Localcopy(origin_addr, - origin_count, - origin_datatype, - req->buffer, - req->origin.dt.size, - MPI_CHAR); - MPID_assert(mpi_errno == MPI_SUCCESS); - } - - pami_result_t rc; - pami_task_t task = MPID_VCR_GET_LPID(win->comm_ptr->vcr, target_rank); - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_START && - !MPIDI_valid_group_rank(task, win->mpid.sync.sc.group)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &req->dest); - MPID_assert(rc == PAMI_SUCCESS); - - - MPIDI_Win_datatype_map(&req->target.dt); - req->result.addr = result_addr; - req->result.count = result_count; - req->result.datatype = result_datatype; - MPIDI_Win_datatype_basic(result_count, result_datatype, &req->result.dt); - MPIDI_Win_datatype_map(&req->result.dt); - req->result_num_contig = req->result.dt.num_contig; - if (win->create_flavor == MPI_WIN_FLAVOR_SHARED) - { - win->mpid.sync.total++; - MPIDI_Fetch_data_op(origin_addr, origin_count, origin_datatype, - result_addr, target_rank, target_disp, - target_count, target_datatype, op, win); - ++win->mpid.sync.complete; - - if (req->buffer_free) { - MPL_free(req->buffer); - MPL_free(req->user_buffer); - req->buffer_free = 0; - } - MPIDI_Win_datatype_unmap(&req->target.dt); - MPIDI_Win_datatype_unmap(&req->result.dt); - - if(req->req_handle) { - MPIR_cc_set(req->req_handle->cc_ptr, 0); - } else { - MPL_free(req); - } - } else { /* non-shared */ - //We wait for #messages depending on target and result_datatype - win->mpid.sync.total += (1 + req->target.dt.num_contig); - { - MPI_Datatype basic_type = MPI_DATATYPE_NULL; - MPIDU_Datatype_get_basic_type(origin_datatype, basic_type); - /* MPIDU_Datatype_get_basic_type() doesn't handle the struct types */ - if ((origin_datatype == MPI_FLOAT_INT) || - (origin_datatype == MPI_DOUBLE_INT) || - (origin_datatype == MPI_LONG_INT) || - (origin_datatype == MPI_SHORT_INT) || - (origin_datatype == MPI_LONG_DOUBLE_INT)) - { - MPID_assert(basic_type == MPI_DATATYPE_NULL); - basic_type = origin_datatype; - } - MPID_assert(basic_type != MPI_DATATYPE_NULL); - - unsigned index; - MPIDI_Win_GetAccMsgInfo * headers = MPL_calloc0(req->target.dt.num_contig, MPIDI_Win_GetAccMsgInfo); - req->accum_headers = headers; - for (index=0; index < req->target.dt.num_contig; ++index) { - headers[index].addr = win->mpid.info[target_rank].base_addr + req->offset + - (size_t)req->target.dt.map[index].DLOOP_VECTOR_BUF; - headers[index].req = req; - headers[index].win = win; - headers[index].type = basic_type; - headers[index].op = op; - headers[index].count = target_count; - headers[index].counter = index; - headers[index].num_contig = req->target.dt.num_contig; - headers[index].size = req->target.dt.size; - headers[index].request = req; - } - - } - - /* The pamid one-sided design requires context post in order to handle the - * case where the number of pending rma operation exceeds the - * 'PAMID_RMA_PENDING' threshold. When there are too many pending requests the - * work function remains on the context post queue (by returning PAMI_EAGAIN) - * so that the next time the context is advanced the work function will be - * invoked again. - * - * TODO - When context post is not required it would be better to attempt a - * direct context operation and then fail over to using context post if - * the rma pending threshold has been reached. This would result in - * better latency for one-sided operations. - */ - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Get_accumulate, req); - } -fn_fail: - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_get_info.c b/src/mpid/pamid/src/onesided/mpid_win_get_info.c deleted file mode 100644 index 5768e99..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_get_info.c +++ /dev/null @@ -1,104 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_get_info.c - * \brief returns a new info object containing the hints of the window - * associated with win. - */ -#include "mpidi_onesided.h" -#include "mpir_info.h" - -/** - * \brief MPI-PAMI glue for MPI_WIN_GET_INFO function - * - * \param[in] win Window - * \param[in] info_p_p Info hint - * \return MPI_SUCCESS - */ -int MPIDI_Win_get_info(MPIR_Win *win, MPIR_Info **info_used) -{ - int mpi_errno = MPI_SUCCESS; - - - /* Populate the predefined info keys */ - if (win->mpid.info_args.no_locks) - mpi_errno = MPIR_Info_set_impl(*info_used, "no_locks", "true"); - else - mpi_errno = MPIR_Info_set_impl(*info_used, "no_locks", "false"); - MPID_assert(mpi_errno == MPI_SUCCESS); - - { -#define BUFSIZE 32 - char buf[BUFSIZE]; - int c = 0; - if (win->mpid.info_args.accumulate_ordering & MPIDI_ACCU_ORDER_RAR) - c += snprintf(buf+c, BUFSIZE-c, "%srar", (c > 0) ? "," : ""); - if (win->mpid.info_args.accumulate_ordering & MPIDI_ACCU_ORDER_RAW) - c += snprintf(buf+c, BUFSIZE-c, "%sraw", (c > 0) ? "," : ""); - if (win->mpid.info_args.accumulate_ordering & MPIDI_ACCU_ORDER_WAR) - c += snprintf(buf+c, BUFSIZE-c, "%swar", (c > 0) ? "," : ""); - if (win->mpid.info_args.accumulate_ordering & MPIDI_ACCU_ORDER_WAW) - c += snprintf(buf+c, BUFSIZE-c, "%swaw", (c > 0) ? "," : ""); - - if (c == 0) { - memcpy(&buf[0],"not set ",10); - } - MPIR_Info_set_impl(*info_used, "accumulate_ordering", buf); - MPID_assert(mpi_errno == MPI_SUCCESS); -#undef BUFSIZE - } - if (win->mpid.info_args.accumulate_ops == (unsigned) MPIDI_ACCU_OPS_SAME_OP) - mpi_errno = MPIR_Info_set_impl(*info_used, "accumulate_ops", "same_op"); - else - mpi_errno = MPIR_Info_set_impl(*info_used, "accumulate_ops", "same_op_no_op"); - - MPID_assert(mpi_errno == MPI_SUCCESS); - - if (win->create_flavor == MPI_WIN_FLAVOR_SHARED) { - if (win->mpid.info_args.alloc_shared_noncontig) - mpi_errno = MPIR_Info_set_impl(*info_used, "alloc_shared_noncontig", "true"); - else - mpi_errno = MPIR_Info_set_impl(*info_used, "alloc_shared_noncontig", "false"); - - MPID_assert(mpi_errno == MPI_SUCCESS); - } - else if (win->create_flavor == MPI_WIN_FLAVOR_ALLOCATE) { - if (win->mpid.info_args.same_size) - mpi_errno = MPIR_Info_set_impl(*info_used, "same_size", "true"); - else - mpi_errno = MPIR_Info_set_impl(*info_used, "same_size", "false"); - - MPID_assert(mpi_errno == MPI_SUCCESS); - } - return mpi_errno; -} - - -int -MPID_Win_get_info(MPIR_Win *win, - MPIR_Info **info_p) -{ - int mpi_errno = MPI_SUCCESS; - - /* Allocate an empty info object */ - mpi_errno = MPIR_Info_alloc(info_p); - MPID_assert(mpi_errno == MPI_SUCCESS); - mpi_errno = MPIDI_Win_get_info(win, info_p); - MPID_assert(mpi_errno == MPI_SUCCESS); - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_lock.c b/src/mpid/pamid/src/onesided/mpid_win_lock.c deleted file mode 100644 index 6d7897c..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_lock.c +++ /dev/null @@ -1,230 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_lock.c - * \brief ??? - */ -#include "mpidi_onesided.h" - -void -MPIDI_WinLockAck_post(pami_context_t context, - unsigned peer, - MPIR_Win * win); - - -void -MPIDI_WinLockAdvance(pami_context_t context, - MPIR_Win * win) -{ - struct MPIDI_Win_sync_lock* slock = &win->mpid.sync.lock; - struct MPIDI_Win_queue* q = &slock->local.requested; - - if ( - (q->head != NULL ) && - ( (slock->local.count == 0) || - ( - (slock->local.type == MPI_LOCK_SHARED) && - (q->head->type == MPI_LOCK_SHARED) - ) - ) - ) - { - struct MPIDI_Win_lock* lock = q->head; - q->head = lock->next; - if (q->head == NULL) - q->tail = NULL; - - ++slock->local.count; - slock->local.type = lock->type; - if (lock->mtype == MPIDI_REQUEST_LOCK) - MPIDI_WinLockAck_post(context, lock->rank, win); - else if (lock->mtype == MPIDI_REQUEST_LOCKALL) - MPIDI_WinLockAllAck_post(context, lock->rank, win); - else - MPID_assert_always(0); - MPL_free(lock); - MPIDI_WinLockAdvance(context, win); - } -} - - -static pami_result_t -MPIDI_WinLockReq_post(pami_context_t context, - void * _info) -{ - MPIDI_WinLock_info* info = (MPIDI_WinLock_info*)_info; - MPIDI_Win_control_t msg = { - .type = MPIDI_WIN_MSGTYPE_LOCKREQ, - .data = { - .lock = { - .type = info->lock_type, - }, - }, - }; - - MPIDI_WinCtrlSend(context, &msg, info->peer, info->win); - return PAMI_SUCCESS; -} - - -void -MPIDI_WinLockReq_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer) -{ - MPIR_Win * win = info->win; - struct MPIDI_Win_lock* lock = MPL_calloc0(1, struct MPIDI_Win_lock); - if (info->type == MPIDI_WIN_MSGTYPE_LOCKREQ) - lock->mtype = MPIDI_REQUEST_LOCK; - else if (info->type == MPIDI_WIN_MSGTYPE_LOCKALLREQ) { - lock->mtype = MPIDI_REQUEST_LOCKALL; - lock->flagAddr = (void *) info->flagAddr; - } - lock->rank = info->rank; - lock->type = info->data.lock.type; - - struct MPIDI_Win_queue* q = &win->mpid.sync.lock.local.requested; - MPID_assert( (q->head != NULL) ^ (q->tail == NULL) ); - if (q->tail == NULL) - q->head = lock; - else - q->tail->next = lock; - q->tail = lock; - - MPIDI_WinLockAdvance(context, win); -} - - -void -MPIDI_WinLockAck_post(pami_context_t context, - unsigned peer, - MPIR_Win * win) -{ - MPIDI_Win_control_t info = { - .type = MPIDI_WIN_MSGTYPE_LOCKACK, - }; - MPIDI_WinCtrlSend(context, &info, peer, win); -} - - -void -MPIDI_WinLockAck_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer) -{ - if (info->type == MPIDI_WIN_MSGTYPE_LOCKACK) - info->win->mpid.sync.lock.remote.locked = 1; - else if (info->type == MPIDI_WIN_MSGTYPE_LOCKALLACK) - info->win->mpid.sync.lock.remote.allLocked += 1; - -} - - -static pami_result_t -MPIDI_WinUnlock_post(pami_context_t context, - void * _info) -{ - MPIDI_WinLock_info* info = (MPIDI_WinLock_info*)_info; - MPIDI_Win_control_t msg = { - .type = MPIDI_WIN_MSGTYPE_UNLOCK, - }; - MPIDI_WinCtrlSend(context, &msg, info->peer, info->win); - info->done = 1; - return PAMI_SUCCESS; -} - - -void -MPIDI_WinUnlock_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer) -{ - MPIR_Win *win = info->win; - --win->mpid.sync.lock.local.count; - MPID_assert((int)win->mpid.sync.lock.local.count >= 0); - MPIDI_WinLockAdvance(context, win); -} - - -int -MPID_Win_lock(int lock_type, - int rank, - int assert, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - struct MPIDI_Win_sync_lock* slock = &win->mpid.sync.lock; - static char FCNAME[] = "MPID_Win_lock"; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_NONE && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_REFENCE){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - if (rank == MPI_PROC_NULL) goto fn_exit; - - MPIDI_WinLock_info info = { - .done = 0, - .peer = rank, - .win = win, - .lock_type = lock_type - }; - - MPIDI_Context_post(MPIDI_Context[0], &info.work, MPIDI_WinLockReq_post, &info); - MPID_PROGRESS_WAIT_WHILE(!slock->remote.locked); -fn_exit: - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_LOCK; - - return mpi_errno; -} - - -int -MPID_Win_unlock(int rank, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_unlock"; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - if (rank == MPI_PROC_NULL) goto fn_exit; - struct MPIDI_Win_sync* sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_DO_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - - MPIDI_WinLock_info info = { - .done = 0, - .peer = rank, - .win = win, - }; - MPIDI_Context_post(MPIDI_Context[0], &info.work, MPIDI_WinUnlock_post, &info); - MPID_PROGRESS_WAIT_WHILE(sync->lock.remote.locked); -fn_exit: - if(win->mpid.sync.target_epoch_type == MPID_EPOTYPE_REFENCE) - { - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_REFENCE; - }else{ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_NONE; - } - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_lock_all.c b/src/mpid/pamid/src/onesided/mpid_win_lock_all.c deleted file mode 100644 index 5dfcd98..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_lock_all.c +++ /dev/null @@ -1,206 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_lock_all.c - * \brief ??? - */ -#include "mpidi_onesided.h" - - -static pami_result_t -MPIDI_WinLockAllReq_post(pami_context_t context, - void * _info) -{ - MPIDI_WinLock_info* info = (MPIDI_WinLock_info*)_info; - MPIDI_Win_control_t msg = { - .type = MPIDI_WIN_MSGTYPE_LOCKALLREQ, - .flagAddr = info, - .data = { - .lock = { - .type = info->lock_type, - }, - }, - }; - - MPIDI_WinCtrlSend(context, &msg, info->peer, info->win); - return PAMI_SUCCESS; -} - - -static pami_result_t -MPIDI_WinUnlockAllReq_post(pami_context_t context, - void * _info) -{ - MPIDI_WinLock_info* info = (MPIDI_WinLock_info*)_info; - MPIDI_Win_control_t msg = { - .type = MPIDI_WIN_MSGTYPE_UNLOCKALL, - .flagAddr = info, - }; - MPIDI_WinCtrlSend(context, &msg, info->peer, info->win); - return PAMI_SUCCESS; -} - -void -MPIDI_WinUnlockAll_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer) -{ - MPIR_Win *win = info->win; - --win->mpid.sync.lock.local.count; - MPID_assert((int)win->mpid.sync.lock.local.count >= 0); - MPIDI_WinLockAdvance(context, win); -} - -void -MPIDI_WinLockAllAck_post(pami_context_t context, - unsigned peer, - MPIR_Win * win) -{ - MPIDI_Win_control_t info = { - .type = MPIDI_WIN_MSGTYPE_LOCKALLACK, - }; - MPIDI_WinCtrlSend(context, &info, peer, win); -} - - -int -MPID_Win_lock_all(int assert, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - int i,size; - MPIDI_WinLock_info *lockQ; - char *cp=NULL; - int nMask,index; - struct MPIDI_Win_sync_lock* slock = &win->mpid.sync.lock; - static char FCNAME[] = "MPID_Win_lock_all"; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_NONE && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_REFENCE){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - size = (MPIR_Comm_size(win->comm_ptr)); - win->mpid.max_ctrlsends = MAX_NUM_CTRLSEND; - nMask= win->mpid.max_ctrlsends - 1; - if ( (cp=getenv("MP_MAX_NUM_CTRLSEND")) ) { - win->mpid.max_ctrlsends = atoi(cp); - } - nMask=(win->mpid.max_ctrlsends - 1); - if (!win->mpid.work.msgQ) { - if (size < (win->mpid.max_ctrlsends)) { - win->mpid.work.msgQ = (void *) MPL_calloc0(size, MPIDI_WinLock_info); - } else { - win->mpid.work.msgQ = (void *) MPL_calloc0((win->mpid.max_ctrlsends), MPIDI_WinLock_info); - } - MPID_assert(win->mpid.work.msgQ != NULL); - win->mpid.work.count=0; - } - lockQ = (MPIDI_WinLock_info *) win->mpid.work.msgQ; - if (size < win->mpid.max_ctrlsends) { - for (i = 0; i < size; i++) { - lockQ[i].done=0; - lockQ[i].peer=i; - lockQ[i].win=win; - lockQ[i].lock_type=MPI_LOCK_SHARED; - MPIDI_Context_post(MPIDI_Context[0], &lockQ[i].work, MPIDI_WinLockAllReq_post, &lockQ[i]); - } - } else { - for (i = 0; i < size; i++) { - if (i < win->mpid.max_ctrlsends) - index=i; - else { - index = i & nMask; - if (!lockQ[index].done) { - MPID_PROGRESS_WAIT_WHILE(lockQ[index].done == 0); - } - } - lockQ[index].done=0; - lockQ[index].peer=i; - lockQ[index].win=win; - lockQ[index].lock_type=MPI_LOCK_SHARED; - MPIDI_Context_post(MPIDI_Context[0], &lockQ[index].work, MPIDI_WinLockAllReq_post, &lockQ[index]); - } - } - /* wait for the lock is granted for all tasks in the window */ - MPID_PROGRESS_WAIT_WHILE(size != slock->remote.allLocked); - - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_LOCK_ALL; - - return mpi_errno; -} - - -int -MPID_Win_unlock_all(MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - int i,size; - MPIDI_WinLock_info *lockQ; - int nMask,index; - struct MPIDI_Win_sync* sync; - static char FCNAME[] = "MPID_Win_unlock_all"; - - if (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL) { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - MPID_assert(win->mpid.work.msgQ != NULL); - lockQ = (MPIDI_WinLock_info *) win->mpid.work.msgQ; - size = MPIR_Comm_size(win->comm_ptr); - nMask = (win->mpid.max_ctrlsends - 1); - if (size < win->mpid.max_ctrlsends) { - for (i = 0; i < size; i++) { - lockQ[i].done=0; - lockQ[i].peer=i; - lockQ[i].win=win; - MPIDI_Context_post(MPIDI_Context[0], &lockQ[i].work, MPIDI_WinUnlockAllReq_post, &lockQ[i]); - } - } else { - for (i = 0; i < size; i++) { - if (i < win->mpid.max_ctrlsends) - index=i; - else { - index = (i & nMask); - if (!lockQ[index].done) { - MPID_PROGRESS_WAIT_WHILE(lockQ[index].done == 0); - } - } - lockQ[index].done=0; - lockQ[index].peer=i; - lockQ[index].win=win; - MPIDI_Context_post(MPIDI_Context[0], &lockQ[index].work, MPIDI_WinUnlockAllReq_post, &lockQ[index]); - } - } - - MPID_PROGRESS_WAIT_WHILE(sync->lock.remote.allLocked); - - if(win->mpid.sync.target_epoch_type == MPID_EPOTYPE_REFENCE) - { - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_REFENCE; - }else{ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_NONE; - } - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_pscw.c b/src/mpid/pamid/src/onesided/mpid_win_pscw.c deleted file mode 100644 index 98d20d5..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_pscw.c +++ /dev/null @@ -1,270 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_pscw.c - * \brief ??? - */ -#include "mpidi_onesided.h" - - -typedef struct -{ - MPIR_Win * win; - - volatile unsigned done; - pami_work_t work; -} MPIDI_WinPSCW_info; - - -static pami_result_t -MPIDI_WinPost_post(pami_context_t context, - void * _info) -{ - MPIDI_WinPSCW_info * info = (MPIDI_WinPSCW_info*)_info; - unsigned peer, index, pid,i; - MPIR_Group *group = info->win->mpid.sync.pw.group; - MPID_assert(group != NULL); - MPIDI_Win_control_t msg = { - .type = MPIDI_WIN_MSGTYPE_POST, - }; - - for (index=0; index < group->size; ++index) { - pid = group->lrank_to_lpid[index].lpid; - for (i=0;i < ((info->win)->comm_ptr->local_size); i++) { - if ((info->win)->comm_ptr->local_group->lrank_to_lpid[i].lpid == pid) { - peer = ((info->win)->comm_ptr->local_group->lrank_to_lpid[i].lrank); - break; - } - } - MPIDI_WinCtrlSend(context, &msg, peer, info->win); - } - - info->done = 1; - return PAMI_SUCCESS; -} - - -void -MPIDI_WinPost_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer) -{ - ++info->win->mpid.sync.pw.count; -} - - -static pami_result_t -MPIDI_WinComplete_post(pami_context_t context, - void * _info) -{ - MPIDI_WinPSCW_info * info = (MPIDI_WinPSCW_info*)_info; - unsigned peer, index,pid,i; - MPIR_Group *group = info->win->mpid.sync.sc.group; - MPID_assert(group != NULL); - MPIDI_Win_control_t msg = { - .type = MPIDI_WIN_MSGTYPE_COMPLETE, - }; - - for (index=0; index < group->size; ++index) { - pid = group->lrank_to_lpid[index].lpid; - for (i=0;i < ((info->win)->comm_ptr->local_size); i++) { - if ((info->win)->comm_ptr->local_group->lrank_to_lpid[i].lpid == pid) { - peer = ((info->win)->comm_ptr->local_group->lrank_to_lpid[i].lrank); - break; - } - } - MPIDI_WinCtrlSend(context, &msg, peer, info->win); - } - - info->done = 1; - return PAMI_SUCCESS; -} - - -void -MPIDI_WinComplete_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer) -{ - ++info->win->mpid.sync.sc.count; -} - - -int -MPID_Win_start(MPIR_Group *group, - int assert, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_start"; - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_NONE && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_REFENCE) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Group_add_ref(group); - - struct MPIDI_Win_sync* sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(group->size != sync->pw.count); - sync->pw.count = 0; - - MPIU_ERR_CHKANDJUMP((win->mpid.sync.sc.group != NULL), mpi_errno, MPI_ERR_GROUP, "**group"); - - win->mpid.sync.sc.group = group; - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_START; - -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - - -int -MPID_Win_complete(MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_complete"; - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_START){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - struct MPIDI_Win_sync* sync = &win->mpid.sync; - MPID_PROGRESS_WAIT_WHILE(sync->total != sync->complete); - sync->total = 0; - sync->started = 0; - sync->complete = 0; - - MPIDI_WinPSCW_info info = { - .done = 0, - .win = win, - }; - MPIDI_Context_post(MPIDI_Context[0], &info.work, MPIDI_WinComplete_post, &info); - MPID_PROGRESS_WAIT_WHILE(!info.done); - - if(win->mpid.sync.target_epoch_type == MPID_EPOTYPE_REFENCE) - { - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_REFENCE; - }else{ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_NONE; - } - - MPIR_Group_release(sync->sc.group); - sync->sc.group = NULL; - return mpi_errno; -} - - -int -MPID_Win_post(MPIR_Group *group, - int assert, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_post"; - if(win->mpid.sync.target_epoch_type != MPID_EPOTYPE_NONE && - win->mpid.sync.target_epoch_type != MPID_EPOTYPE_REFENCE){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Group_add_ref(group); - - MPIU_ERR_CHKANDJUMP((win->mpid.sync.pw.group != NULL), mpi_errno, MPI_ERR_GROUP, "**group"); - - win->mpid.sync.pw.group = group; - - MPIDI_WinPSCW_info info = { - .done = 0, - .win = win, - }; - MPIDI_Context_post(MPIDI_Context[0], &info.work, MPIDI_WinPost_post, &info); - MPID_PROGRESS_WAIT_WHILE(!info.done); - - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_POST; - -fn_fail: - return mpi_errno; -} - - -int -MPID_Win_wait(MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_wait"; - struct MPIDI_Win_sync* sync = &win->mpid.sync; - - if(win->mpid.sync.target_epoch_type != MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Group *group = sync->pw.group; - MPID_PROGRESS_WAIT_WHILE(group->size != sync->sc.count); - sync->sc.count = 0; - sync->pw.group = NULL; - - MPIR_Group_release(group); - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_REFENCE; - }else{ - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_NONE; - } - return mpi_errno; -} - - -int -MPID_Win_test(MPIR_Win *win, - int *flag) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_test"; - struct MPIDI_Win_sync* sync = &win->mpid.sync; - - if(win->mpid.sync.target_epoch_type != MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Group *group = sync->pw.group; - if (group->size == sync->sc.count) - { - sync->sc.count = 0; - sync->pw.group = NULL; - *flag = 1; - MPIR_Group_release(group); - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_REFENCE; - }else{ - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_NONE; - } - } - else - { - *flag = 0; - MPID_Progress_poke(); - } - - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_put.c b/src/mpid/pamid/src/onesided/mpid_win_put.c deleted file mode 100644 index 9e70f56..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_put.c +++ /dev/null @@ -1,434 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_put.c - * \brief ??? - */ -#include "mpidi_onesided.h" -#include "mpidi_util.h" - - -static inline int -MPIDI_Put_use_pami_rput(pami_context_t context, MPIDI_Win_request * req) -__attribute__((__always_inline__)); -#ifdef RDMA_FAILOVER -static inline int -MPIDI_Put_use_pami_put(pami_context_t context, MPIDI_Win_request * req) -__attribute__((__always_inline__)); -#endif - - -static pami_result_t -MPIDI_Put(pami_context_t context, - void * _req) -{ - MPIDI_Win_request *req = (MPIDI_Win_request*)_req; - pami_result_t rc; - -#ifdef USE_PAMI_RDMA - rc = MPIDI_Put_use_pami_rput(context, req); -#else - if( (req->origin.memregion_used) && (req->win->mpid.info[req->target.rank].memregion_used) ) - { - rc = MPIDI_Put_use_pami_rput(context, req); - } else { - rc = MPIDI_Put_use_pami_put(context, req); - } -#endif - if( rc == PAMI_EAGAIN) - return rc; - - - return PAMI_SUCCESS; -} - - -static inline int -MPIDI_Put_use_pami_rput(pami_context_t context, MPIDI_Win_request * req) -{ - int use_typed_rdma = 0; - if (!req->target.dt.contig || !req->origin.dt.contig) { - use_typed_rdma = 0; - if (MPIDI_Process.typed_onesided == 1) - use_typed_rdma = 1; - } - - if (use_typed_rdma) { - pami_result_t rc; - pami_rput_typed_t params; - /* params need to zero out to avoid passing garbage to PAMI */ - params=zero_rput_typed_parms; - - params.rma.dest=req->dest; - params.rma.hints.buffer_registered = PAMI_HINT_ENABLE; - params.rma.hints.use_rdma = PAMI_HINT_ENABLE; - params.rma.bytes = req->target.dt.size; - params.rma.cookie = req; - params.rma.done_fn = NULL; - params.rdma.local.mr=&req->origin.memregion; - params.rdma.remote.mr=&req->win->mpid.info[req->target.rank].memregion; - params.rdma.remote.offset= req->offset; - params.rdma.local.offset = req->state.local_offset; - params.put.rdone_fn= MPIDI_Win_DoneCB; - - params.type.local = *(pami_type_t *)(req->origin.dt.pointer->device_datatype); - params.type.remote = *(pami_type_t *)(req->target.dt.pointer->device_datatype); - - rc = PAMI_Rput_typed(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - - } - else { - pami_result_t rc; - pami_rput_simple_t params; - /* params need to zero out to avoid passing garbage to PAMI */ - params=zero_rput_parms; - - params.rma.dest=req->dest; - params.rma.hints.buffer_registered = PAMI_HINT_ENABLE; - params.rma.hints.use_rdma = PAMI_HINT_ENABLE; - params.rma.bytes = 0; - params.rma.cookie = req; - params.rma.done_fn = NULL; - params.rdma.local.mr=&req->origin.memregion; - params.rdma.remote.mr=&req->win->mpid.info[req->target.rank].memregion; - params.rdma.remote.offset= req->offset; - params.put.rdone_fn= MPIDI_Win_DoneCB; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - TRACE_ERR("Start index=%u/%d l-addr=%p r-base=%p r-offset=%zu (sync->started=%u sync->complete=%u)\n", - req->state.index, req->target.dt.num_contig, req->buffer, req->win->mpid.info[req->target.rank].base_addr, req->offset, sync->started, sync->complete); - while (req->state.index < req->target.dt.num_contig) { - if (sync->started > sync->complete + MPIDI_Process.rma_pending) - { - TRACE_ERR("Bailing out; index=%u/%d sync->started=%u sync->complete=%u\n", - req->state.index, req->target.dt.num_contig, sync->started, sync->complete); - return PAMI_EAGAIN; - } - ++sync->started; - - - params.rma.bytes = req->target.dt.map[req->state.index].DLOOP_VECTOR_LEN; - params.rdma.remote.offset = req->offset + (size_t)req->target.dt.map[req->state.index].DLOOP_VECTOR_BUF; - params.rdma.local.offset = req->state.local_offset; -#ifdef TRACE_ON - unsigned* buf = (unsigned*)(req->buffer + params.rdma.local.offset); -#endif - TRACE_ERR(" Sub index=%u bytes=%zu l-offset=%zu r-offset=%zu buf=%p *(int*)buf=0x%08x\n", - req->state.index, params.rma.bytes, params.rdma.local.offset, params.rdma.remote.offset, buf, *buf); - - /** sync->total will be updated with every RMA and the complete - will not change till that RMA has completed. In the meanwhile - the rest of the RMAs will have memory leaks */ - if (req->target.dt.num_contig - req->state.index == 1) { - rc = PAMI_Rput(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; - } else { - rc = PAMI_Rput(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - req->state.local_offset += params.rma.bytes; - ++req->state.index; - } - } - } - return PAMI_SUCCESS; -} - -#ifdef RDMA_FAILOVER -static inline int -MPIDI_Put_use_pami_put(pami_context_t context, MPIDI_Win_request * req) -{ - pami_result_t rc; - pami_put_simple_t params; - - params = zero_put_parms; - - params.rma.dest=req->dest; - params.rma.hints.use_rdma = PAMI_HINT_DEFAULT; -#ifndef OUT_OF_ORDER_HANDLING - params.rma.hints.no_long_header= 1, -#endif - params.rma.bytes = 0; - params.rma.cookie = req; - params.rma.done_fn = NULL; - params.addr.local=req->buffer; - params.addr.remote=req->win->mpid.info[req->target.rank].base_addr; - params.put.rdone_fn= MPIDI_Win_DoneCB; - - struct MPIDI_Win_sync* sync = &req->win->mpid.sync; - TRACE_ERR("Start index=%u/%d l-addr=%p r-base=%p r-offset=%zu (sync->started=%u sync->complete=%u)\n", - req->state.index, req->target.dt.num_contig, req->buffer, req->win->mpid.info[req->target.rank].base_addr, req->offset, sync->started, sync->complete); - while (req->state.index < req->target.dt.num_contig) { - if (sync->started > sync->complete + MPIDI_Process.rma_pending) - { - TRACE_ERR("Bailing out; index=%u/%d sync->started=%u sync->complete=%u\n", - req->state.index, req->target.dt.num_contig, sync->started, sync->complete); - return PAMI_EAGAIN; - } - ++sync->started; - - - params.rma.bytes = req->target.dt.map[req->state.index].DLOOP_VECTOR_LEN; - params.addr.local = req->buffer+req->state.local_offset; - params.addr.remote = req->win->mpid.info[req->target.rank].base_addr+ req->offset + (size_t)req->target.dt.map[req->state.index].DLOOP_VECTOR_BUF; -#ifdef TRACE_ON - unsigned* buf = (unsigned*)(req->buffer + req->state.local_offset); -#endif - TRACE_ERR(" Sub index=%u bytes=%zu l-offset=%zu r-offset=%zu buf=%p *(int*)buf=0x%08x\n", - req->state.index, params.rma.bytes, params.addr.local, params.addr.remote, buf, *buf); - - /** sync->total will be updated with every RMA and the complete - will not change till that RMA has completed. In the meanwhile - the rest of the RMAs will have memory leaks */ - if (req->target.dt.num_contig - req->state.index == 1) { - rc = PAMI_Put(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - return PAMI_SUCCESS; - } else { - rc = PAMI_Put(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - req->state.local_offset += params.rma.bytes; - ++req->state.index; - } - } - return PAMI_SUCCESS; -} -#endif - - -/** - * \brief MPI-PAMI glue for MPI_PUT function - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Put -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Put(const void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - int shm_locked=0; - void * target_addr; - MPIDI_Win_request *req = MPL_calloc0(1, MPIDI_Win_request); - req->win = win; - if(win->mpid.request_based != 1) - req->type = MPIDI_WIN_REQUEST_PUT; - else { - req->req_handle = win->mpid.rreq; - req->type = MPIDI_WIN_REQUEST_RPUT; - req->req_handle->mpid.win_req = req; - } - - if(win->mpid.sync.origin_epoch_type == win->mpid.sync.target_epoch_type && - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_REFENCE){ - win->mpid.sync.origin_epoch_type = MPID_EPOTYPE_FENCE; - win->mpid.sync.target_epoch_type = MPID_EPOTYPE_FENCE; - } - - if(win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_NONE || - win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_POST){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - req->offset = target_disp * win->mpid.info[target_rank].disp_unit; -#ifdef __BGQ__ - /* PAMI limitation as it doesnt permit VA of 0 to be passed into - * memregion create, so we must pass base_va of heap computed from - * an SPI call instead. So the target offset must be adjusted */ - if (req->win->create_flavor == MPI_WIN_FLAVOR_DYNAMIC) - req->offset -= (size_t)req->win->mpid.info[target_rank].base_addr; -#endif - - MPIDI_Win_datatype_basic(origin_count, - origin_datatype, - &req->origin.dt); - MPIDI_Win_datatype_basic(target_count, - target_datatype, - &req->target.dt); - #ifndef MPIDI_NO_ASSERT - MPID_assert(req->origin.dt.size == req->target.dt.size); - #else - MPIU_ERR_CHKANDJUMP((req->origin.dt.size != req->target.dt.size), mpi_errno, MPI_ERR_SIZE, "**rmasize"); - #endif - - - - if ( (req->origin.dt.size == 0) || - (target_rank == MPI_PROC_NULL)) - { - if(req->req_handle) - MPIR_cc_set(req->req_handle->cc_ptr, 0); - else - MPL_free(req); - return MPI_SUCCESS; - } - - - if ((target_rank == win->comm_ptr->rank) || (win->create_flavor == MPI_WIN_FLAVOR_SHARED)) - { - size_t offset = req->offset; - if (target_rank == win->comm_ptr->rank) - target_addr = win->base + req->offset; - else - target_addr = win->mpid.info[target_rank].base_addr + req->offset; - - mpi_errno = MPIR_Localcopy(origin_addr, - origin_count, - origin_datatype, - target_addr, - target_count, - target_datatype); - - /* The instant this completion counter is set to zero another thread - * may notice the change and begin freeing request resources. The - * thread executing the code in this function must not touch any - * portion of the request structure after decrementing the completion - * counter. - * - * See MPID_Request_free_inline() - */ - if(req->req_handle) - MPIR_cc_set(req->req_handle->cc_ptr, 0); - else - MPL_free(req); - return mpi_errno; - } - req->target.rank = target_rank; - - - /* Only pack the origin data if the origin is non-contiguous and we are using the simple PAMI_Rput. - * If we are using the typed PAMI_Rput_typed use the origin address as-is, if we are using the simple - * PAMI_Rput with contiguous data use the origin address with the lower-bound adjustment. - */ - if (req->origin.dt.contig || (!req->origin.dt.contig && (MPIDI_Process.typed_onesided == 1))) - { - req->buffer_free = 0; - if ((req->origin.dt.contig && req->target.dt.contig && (MPIDI_Process.typed_onesided == 1)) || (!(MPIDI_Process.typed_onesided == 1))) - req->buffer = (void *) ((uintptr_t) origin_addr + req->origin.dt.true_lb); - else - req->buffer = (void *) ((uintptr_t) origin_addr); - } - else - { - req->buffer_free = 1; - req->buffer = MPL_malloc(req->origin.dt.size); - MPID_assert(req->buffer != NULL); - - int mpi_errno = 0; - mpi_errno = MPIR_Localcopy(origin_addr, - origin_count, - origin_datatype, - req->buffer, - req->origin.dt.size, - MPI_CHAR); - MPID_assert(mpi_errno == MPI_SUCCESS); - } - - - pami_result_t rc; - pami_task_t task = MPID_VCR_GET_LPID(win->comm_ptr->vcr, target_rank); - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_START && - !MPIDI_valid_group_rank(task, win->mpid.sync.sc.group)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - rc = PAMI_Endpoint_create(MPIDI_Client, task, 0, &req->dest); - MPID_assert(rc == PAMI_SUCCESS); - -#ifdef USE_PAMI_RDMA - size_t length_out; - rc = PAMI_Memregion_create(MPIDI_Context[0], - req->buffer, - req->origin.dt.size, - &length_out, - &req->origin.memregion); - MPID_assert(rc == PAMI_SUCCESS); - MPID_assert(req->origin.dt.size == length_out); -#else - if(!MPIDI_Process.mp_s_use_pami_get) - { - size_t length_out; - rc = PAMI_Memregion_create(MPIDI_Context[0], - req->buffer, - req->origin.dt.size, - &length_out, - &req->origin.memregion); - if(rc == PAMI_SUCCESS) { - MPID_assert(req->origin.dt.size == length_out); - req->origin.memregion_used = 1; - } - } -#endif - - - if ((!req->target.dt.contig || !req->origin.dt.contig) && (MPIDI_Process.typed_onesided == 1)) - /* If the datatype is non-contiguous and the PAMID typed_onesided optimization - * is enabled then we will be using the typed interface and will only make 1 call. - */ - win->mpid.sync.total = 1; - else { - MPIDI_Win_datatype_map(&req->target.dt); - win->mpid.sync.total += req->target.dt.num_contig; - } - if ((MPIDI_Process.typed_onesided == 1) && (!req->target.dt.contig || !req->origin.dt.contig)) { - /* We will use the PAMI_Rput_typed call so we need to make sure any MPI_Type_free before the context - * executes the put does not free the MPIDU_Datatype, which would also free the associated PAMI datatype which - * is still needed for communication -- decrement the ref in the callback to allow the MPIR_Datatype - * to be freed once the PAMI communication has completed. - */ - MPIDU_Datatype_add_ref(req->origin.dt.pointer); - MPIDU_Datatype_add_ref(req->target.dt.pointer); - } - /* The pamid one-sided design requires context post in order to handle the - * case where the number of pending rma operation exceeds the - * 'PAMID_RMA_PENDING' threshold. When there are too many pending requests the - * work function remains on the context post queue (by returning PAMI_EAGAIN) - * so that the next time the context is advanced the work function will be - * invoked again. - * - * TODO - When context post is not required it would be better to attempt a - * direct context operation and then fail over to using context post if - * the rma pending threshold has been reached. This would result in - * better latency for one-sided operations. - */ - PAMI_Context_post(MPIDI_Context[0], &req->post_request, MPIDI_Put, req); - -fn_fail: - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_reqops.c b/src/mpid/pamid/src/onesided/mpid_win_reqops.c deleted file mode 100644 index 36518d1..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_reqops.c +++ /dev/null @@ -1,253 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_reqops.c - * \brief ??? - */ -#include "mpidi_onesided.h" - - -/** - * \brief MPI-PAMI glue for MPI_PUT function - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Rput -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Rput(const void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPIR_Win *win, - MPIR_Request **request) -{ - int mpi_errno = MPI_SUCCESS; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Request *rreq = MPIDI_Request_create1(); - *request = rreq; - rreq->kind = MPIR_REQUEST_KIND__RMA; - win->mpid.rreq = rreq; - win->mpid.request_based = 1; - - /* Enqueue or perform the RMA operation */ - mpi_errno = MPID_Put(origin_addr, origin_count, - origin_datatype, target_rank, - target_disp, target_count, - target_datatype, win); - MPID_assert(mpi_errno == MPI_SUCCESS); - return (mpi_errno); -} - - -/** - * \brief MPI-PAMI glue for MPI_GET function - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Rget -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Rget(void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPIR_Win *win, - MPIR_Request **request) -{ - int mpi_errno = MPI_SUCCESS; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Request *rreq = MPIDI_Request_create1(); - rreq->kind = MPIR_REQUEST_KIND__RMA; - *request = rreq; - win->mpid.rreq = rreq; - win->mpid.request_based = 1; - - mpi_errno = MPID_Get(origin_addr, origin_count, - origin_datatype, target_rank, - target_disp, target_count, - target_datatype, win); - MPID_assert(mpi_errno == MPI_SUCCESS); - return(mpi_errno); -} - -/** - * \brief MPI-PAMI glue for MPI_ACCUMULATE function - * - * According to the MPI Specification: - * - * Each datatype argument must be a predefined datatype or - * a derived datatype, where all basic components are of the - * same predefined datatype. Both datatype arguments must be - * constructed from the same predefined datatype. - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] op Operand to perform - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Raccumulate -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Raccumulate(const void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPI_Op op, - MPIR_Win *win, - MPIR_Request **request) -{ - int mpi_errno = MPI_SUCCESS; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Request *rreq = MPIDI_Request_create1(); - rreq->kind = MPIR_REQUEST_KIND__RMA; - *request = rreq; - win->mpid.rreq = rreq; - win->mpid.request_based = 1; - - /* Enqueue or perform the RMA operation */ - mpi_errno = MPID_Accumulate(origin_addr, origin_count, - origin_datatype, target_rank, - target_disp, target_count, - target_datatype, op, win); - MPID_assert(mpi_errno == MPI_SUCCESS); - return(mpi_errno); -} - - -/** - * \brief MPI-PAMI glue for MPI_ACCUMULATE function - * - * According to the MPI Specification: - * - * Each datatype argument must be a predefined datatype or - * a derived datatype, where all basic components are of the - * same predefined datatype. Both datatype arguments must be - * constructed from the same predefined datatype. - * - * \param[in] origin_addr Source buffer - * \param[in] origin_count Number of datatype elements - * \param[in] origin_datatype Source datatype - * \param[in] target_rank Destination rank (target) - * \param[in] target_disp Displacement factor in target buffer - * \param[in] target_count Number of target datatype elements - * \param[in] target_datatype Destination datatype - * \param[in] op Operand to perform - * \param[in] win Window - * \return MPI_SUCCESS - */ -#undef FUNCNAME -#define FUNCNAME MPID_Rget_accumulate -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int -MPID_Rget_accumulate(const void *origin_addr, - int origin_count, - MPI_Datatype origin_datatype, - void *result_addr, - int result_count, - MPI_Datatype result_datatype, - int target_rank, - MPI_Aint target_disp, - int target_count, - MPI_Datatype target_datatype, - MPI_Op op, - MPIR_Win *win, - MPIR_Request **request) -{ - - int mpi_errno = MPI_SUCCESS; - - if(win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK && - win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL){ - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - - MPIR_Request *rreq = MPIDI_Request_create1(); - rreq->kind = MPIR_REQUEST_KIND__RMA; - *request = rreq; - win->mpid.rreq = rreq; - win->mpid.request_based = 1; - - /* Enqueue or perform the RMA operation */ - mpi_errno = MPID_Get_accumulate(origin_addr, origin_count, - origin_datatype, result_addr, - result_count, result_datatype, - target_rank, target_disp, - target_count, target_datatype, - op, win); - MPID_assert(mpi_errno == MPI_SUCCESS); - return (mpi_errno); -} - diff --git a/src/mpid/pamid/src/onesided/mpid_win_set_info.c b/src/mpid/pamid/src/onesided/mpid_win_set_info.c deleted file mode 100644 index 78ce569..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_set_info.c +++ /dev/null @@ -1,102 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_set_info.c - * \brief sets new values for the hints of the window of the window - * associated with win. - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_WIN_SET_INFO function - * - * \param[in] win Window - * \param[in] info Info hint - * \return MPI_SUCCESS - */ - -int MPIDI_Win_set_info(MPIR_Win *win, MPIR_Info *info) -{ - - int mpi_errno = MPI_SUCCESS; - MPIR_Info *curr_ptr; - char *value, *token; - char *savePtr; - curr_ptr = info->next; - uint save_ordering; - - while (curr_ptr) { - if (!strcmp(curr_ptr->key,"no_locks")) { - if (!strcmp(curr_ptr->value,"true")) { - win->mpid.info_args.no_locks=1; - } else - win->mpid.info_args.no_locks=0; - } else if (!strcmp(curr_ptr->key,"accumulate_ordering")) - { - save_ordering=(uint) win->mpid.info_args.accumulate_ordering; - win->mpid.info_args.accumulate_ordering=0; - value = curr_ptr->value; - token = (char *) strtok_r(value,"," , &savePtr); - while (token) { - if (!memcmp(token,"rar",3)) - { - win->mpid.info_args.accumulate_ordering = - (win->mpid.info_args.accumulate_ordering | MPIDI_ACCU_ORDER_RAR); - } else if (!memcmp(token,"raw",3)) - { - win->mpid.info_args.accumulate_ordering = - (win->mpid.info_args.accumulate_ordering | MPIDI_ACCU_ORDER_RAW); - } else if (!memcmp(token,"war",3)) - { - win->mpid.info_args.accumulate_ordering = - (win->mpid.info_args.accumulate_ordering | MPIDI_ACCU_ORDER_WAR); - } else if (!memcmp(token,"waw",3)) - { - win->mpid.info_args.accumulate_ordering = - (win->mpid.info_args.accumulate_ordering | MPIDI_ACCU_ORDER_WAW); - } else - MPID_assert_always(0); - token = (char *) strtok_r(NULL,"," , &savePtr); - } - if (win->mpid.info_args.accumulate_ordering == 0) { - win->mpid.info_args.accumulate_ordering= - (MPIDI_Win_info_accumulate_ordering) save_ordering; - } - } else if (!strcmp(curr_ptr->key,"accumulate_ops")) - { - /* the default setting is MPIDI_ACCU_SAME_OP_NO_OP */ - if (!strcmp(curr_ptr->value,"same_op")) - win->mpid.info_args.accumulate_ops = MPIDI_ACCU_SAME_OP; - } - curr_ptr = curr_ptr->next; - } - - return mpi_errno; -} - -int -MPID_Win_set_info(MPIR_Win *win, MPIR_Info *info) -{ - int mpi_errno = MPI_SUCCESS; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - - mpi_errno = MPIDI_Win_set_info(win, info); - MPID_assert(mpi_errno == MPI_SUCCESS); - mpi_errno = MPIR_Barrier_impl(win->comm_ptr, &errflag); - return mpi_errno; -} diff --git a/src/mpid/pamid/src/onesided/mpid_win_shared_query.c b/src/mpid/pamid/src/onesided/mpid_win_shared_query.c deleted file mode 100644 index 30c5cd9..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_shared_query.c +++ /dev/null @@ -1,65 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_shared_query.c - * \brief queries the process-local address for remote memory segments - * created with MPI_Win_allocate_shared. - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_shared_query function - * - * Query the size and base pointer for a patch of a shared memory window - * - * \param[in] win shared memory window object - * \param[in] rank rank in the group of window win or MPI_PROC_NULL - * \param[out] size size of the window segment (non-negative integer) - * \param[out] disp_unit local unit size for displacements, in bytes - * \param[out] base_ptr address for load/store access to window segment - * \return MPI_SUCCESS, MPI_ERR_OTHER, or error returned from - */ - -int -MPID_Win_shared_query(MPIR_Win *win, int rank, MPI_Aint *size, - int *disp_unit, void *base_ptr) -{ - int i, mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_shared_query"; - MPIU_ERR_CHKANDSTMT((win->create_flavor != MPI_WIN_FLAVOR_SHARED), mpi_errno, - MPI_ERR_RMA_FLAVOR, return mpi_errno, "**rmaflavor"); - - if (rank == MPI_PROC_NULL) { - for (i=0; i<win->comm_ptr->local_size; ++i) { - if (win->mpid.info[i].base_size != 0) { - *size = win->mpid.info[i].base_size; - *disp_unit = win->mpid.info[i].disp_unit; - break; - } - } - *((void **) base_ptr) = win->base; /* return the begin. address of shared segment */ - } else { - *((void **) base_ptr) = win->mpid.info[rank].base_addr; /* return the begin. address of the rank */ - *size = win->mpid.info[rank].base_size; - *disp_unit = win->mpid.info[rank].disp_unit; - } - - - return mpi_errno; -} - diff --git a/src/mpid/pamid/src/onesided/mpid_win_sync.c b/src/mpid/pamid/src/onesided/mpid_win_sync.c deleted file mode 100644 index d014602..0000000 --- a/src/mpid/pamid/src/onesided/mpid_win_sync.c +++ /dev/null @@ -1,54 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpid_win_sync.c - * \brief returns a new info object containing the hints of the window - * associated with win. - */ -#include "mpidi_onesided.h" - -/** - * \brief MPI-PAMI glue for MPI_Win_sync function - * - * The funcion synchronizes the private and public window copies of win. - * For the purpose of synchronizing the private and public window, - * MPI_Win_sync has the effect of ending and reopening an access and - * exposure epoch on the window. (note that it does not actually end an - * epoch or complete any pending MPI RMA operations). - * - * \param[in] win window object - * \return MPI_SUCCESS, MPI_ERR_RMA_SYNC - */ - - -int -MPID_Win_sync(MPIR_Win *win) -{ - int mpi_errno = MPI_SUCCESS; - static char FCNAME[] = "MPID_Win_flush"; - - if((win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK) && - (win->mpid.sync.origin_epoch_type != MPID_EPOTYPE_LOCK_ALL)) - { - MPIU_ERR_SETANDSTMT(mpi_errno, MPI_ERR_RMA_SYNC, - return mpi_errno, "**rmasync"); - } - OPA_read_write_barrier(); - return mpi_errno; -} - diff --git a/src/mpid/pamid/src/onesided/mpidi_onesided.h b/src/mpid/pamid/src/onesided/mpidi_onesided.h deleted file mode 100644 index 369866f..0000000 --- a/src/mpid/pamid/src/onesided/mpidi_onesided.h +++ /dev/null @@ -1,389 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpidi_onesided.h - * \brief ??? - */ -#include <mpidimpl.h> -#include "opa_primitives.h" - -#ifndef __src_onesided_mpidi_onesided_h__ -#define __src_onesided_mpidi_onesided_h__ - -pami_rget_simple_t zero_rget_parms; -pami_get_simple_t zero_get_parms; -pami_rput_simple_t zero_rput_parms; -pami_put_simple_t zero_put_parms; -pami_rput_typed_t zero_rput_typed_parms; -pami_rget_typed_t zero_rget_typed_parms; -pami_send_t zero_send_parms; -pami_send_immediate_t zero_send_immediate_parms; -pami_recv_t zero_recv_parms; -pami_rmw_t zero_rmw_parms; - -#define MPIDI_QUICKSLEEP usleep(1); -#define MAX_NUM_CTRLSEND 1024 /* no more than 1024 outstanding control sends */ - -#define MPIDI_SHM_MUTEX_LOCK(win) { \ - int mpi_errno=0; \ - do { \ - pthread_mutex_t *shm_mutex = (pthread_mutex_t *) &win->mpid.shm->ctrl->mutex_lock; \ - int rval = pthread_mutex_lock(shm_mutex); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_lock", \ - "**pthread_lock %s", strerror(rval)); \ - } while (0); } - - -#define MPIDI_SHM_MUTEX_UNLOCK(win) { \ - int mpi_errno=0; \ - do { \ - pthread_mutex_t *shm_mutex = (pthread_mutex_t *) &win->mpid.shm->ctrl->mutex_lock; \ - int rval = pthread_mutex_unlock(shm_mutex); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_unlock", \ - "**pthread_unlock %s", strerror(rval)); \ - } while (0); } - - -#define MPIDI_SHM_MUTEX_INIT(win) \ - do { \ - int rval=0; \ - int mpi_errno=0; \ - pthread_mutexattr_t attr; \ - pthread_mutex_t *shm_mutex = (pthread_mutex_t *) &win->mpid.shm->ctrl->mutex_lock; \ - \ - rval = pthread_mutexattr_init(&attr); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_mutex", \ - "**pthread_mutex %s", strerror(rval)); \ - rval = pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_mutex", \ - "**pthread_mutex %s", strerror(rval)); \ - rval = pthread_mutex_init(shm_mutex, &attr); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_mutex", \ - "**pthread_mutex %s", strerror(rval)); \ - rval = pthread_mutexattr_destroy(&attr); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_mutex", \ - "**pthread_mutex %s", strerror(rval)); \ - } while (0); - -#define MPIDI_SHM_MUTEX_DESTROY(win) { \ - int mpi_errno=0; \ - do { \ - pthread_mutex_t *shm_mutex = (pthread_mutex_t *) &(win)->mpid.shm->ctrl->mutex_lock;\ - int rval = pthread_mutex_destroy(shm_mutex); \ - MPIU_ERR_CHKANDJUMP1(rval, mpi_errno, MPI_ERR_OTHER, "**pthread_mutex", \ - "**pthread_mutex %s", strerror(rval)); \ - } while (0); } - -#define MPIDI_ACQUIRE_SHARED_LOCK(win) { \ - MPIDI_SHM_MUTEX_LOCK(win); \ - OPA_fetch_and_add_int((OPA_int_t *)&win->mpid.shm->ctrl->active,(int) 1); \ -} - -#define MPIDI_RELEASE_SHARED_LOCK(win) { \ - MPIDI_SHM_MUTEX_UNLOCK(win); \ - OPA_fetch_and_add_int((OPA_int_t *)&(win->mpid.shm->ctrl->active),(int) -1); \ -} - - - - -/** - * \brief One-sided Message Types - */ -typedef enum - { - MPIDI_WIN_MSGTYPE_LOCKACK, /**< Lock acknowledge */ - MPIDI_WIN_MSGTYPE_LOCKALLACK, /**< Lock all acknowledge */ - MPIDI_WIN_MSGTYPE_LOCKREQ, /**< Lock window */ - MPIDI_WIN_MSGTYPE_LOCKALLREQ, /**< Lock all window */ - MPIDI_WIN_MSGTYPE_UNLOCK, /**< Unlock window */ - MPIDI_WIN_MSGTYPE_UNLOCKALL, /**< Unlock window */ - - MPIDI_WIN_MSGTYPE_COMPLETE, /**< End a START epoch */ - MPIDI_WIN_MSGTYPE_POST, /**< Begin POST epoch */ - } MPIDI_Win_msgtype_t; - -typedef enum - { - MPIDI_WIN_REQUEST_ACCUMULATE, - MPIDI_WIN_REQUEST_GET, - MPIDI_WIN_REQUEST_PUT, - MPIDI_WIN_REQUEST_GET_ACCUMULATE, - MPIDI_WIN_REQUEST_RACCUMULATE, - MPIDI_WIN_REQUEST_RGET, - MPIDI_WIN_REQUEST_RPUT, - MPIDI_WIN_REQUEST_RGET_ACCUMULATE, - MPIDI_WIN_REQUEST_COMPARE_AND_SWAP, - MPIDI_WIN_REQUEST_FETCH_AND_OP, - } MPIDI_Win_requesttype_t; - -typedef enum - { - MPIDI_ACCU_OPS_SAME_OP, - MPIDI_ACCU_OPS_SAME_OP_NO_OP - } MPIDI_Win_info_arg_vals_accumulate_ops; - - -typedef struct -{ - MPIDI_Win_msgtype_t type; - MPIR_Win *win; - int rank; /* MPI rank */ - void *flagAddr; - union - { - struct - { - int type; - } lock; - } data; -} MPIDI_Win_control_t; - - -#define MAX_ATOMIC_TYPE_SIZE 32 -typedef struct -{ - char buf[MAX_ATOMIC_TYPE_SIZE]; //Origin value or ack result value - char test[MAX_ATOMIC_TYPE_SIZE]; //Test element for CAS - void * result_addr; //Address on source to store output - void * remote_addr; //Address of target on destination - void * request_addr; //Address of the request object - MPI_Datatype datatype; - MPI_Op op; - int atomic_type; -} MPIDI_AtomicHeader_t; - - -typedef struct MPIDI_WinLock_info -{ - unsigned peer; - int lock_type; - struct MPIR_Win * win; - void *flagAddr; - volatile unsigned done; - pami_work_t work; -} MPIDI_WinLock_info; - - - -typedef struct -{ - MPIDU_Datatype* pointer; - MPI_Datatype type; - int count; - int contig; - MPI_Aint true_lb; - intptr_t size; - - int num_contig; - DLOOP_VECTOR * map; - DLOOP_VECTOR __map; -} MPIDI_Datatype; - - -typedef struct -{ - void * addr; - void * result_addr; /* anchor the result buffer address*/ - void * req; - MPIR_Win * win; - MPI_Datatype type; - MPI_Op op; - pami_endpoint_t origin_endpoint; /* needed for returning the result */ - /* for MPI_Get_accumulate */ - size_t len; /* length of the send data */ -} MPIDI_Win_MsgInfo; - -typedef struct -{ - void * addr; - void * req; - MPIR_Win * win; - MPI_Datatype type; - MPI_Op op; - int count; - int counter; - int num_contig; - int size; - void * request; - void * tptr; - pami_endpoint_t src_endpoint; -} MPIDI_Win_GetAccMsgInfo; - - -/** \todo make sure that the extra fields are removed */ -typedef struct MPIDI_Win_request -{ - MPIR_Win *win; - MPIDI_Win_requesttype_t type; - pami_endpoint_t dest; - size_t offset; - pami_work_t post_request; - - struct - { - unsigned index; - size_t local_offset; - } state; - - void * accum_headers; - - struct - { - pami_memregion_t memregion; -#ifdef RDMA_FAILOVER - uint32_t memregion_used; /**< memory region registered or not */ -#endif - void *addr; - void *result_addr; /** result buffer that holds target buffer data */ - int count; - MPI_Datatype datatype; - MPIDI_Datatype dt; - int completed; - } origin; - - struct - { - pami_task_t rank; /**< Comm-local rank */ - MPIDI_Datatype dt; - } target; - - struct - { - void *addr; - int count; - MPI_Datatype datatype; - MPIDI_Datatype dt; - } result; - - void *user_buffer; - void *compare_buffer; /* anchor of compare buffer for compare and swap */ - uint32_t buffer_free; - void *buffer; - struct MPIDI_Win_request *next; - MPI_Op op; - int result_num_contig; - - - /* for RMA atomic functions */ - - pami_atomic_t pami_op; - pami_type_t pami_datatype; - - int request_based; /* flag for request based rma */ - MPIR_Request *req_handle; /* anchor of MPIR_Request struc for request based rma*/ -} MPIDI_Win_request; - -MPIDI_Win_request zero_req; /* used for init. request structure to 0 */ - -void -MPIDI_Win_datatype_basic(int count, - MPI_Datatype datatype, - MPIDI_Datatype * dt); -void -MPIDI_Win_datatype_map (MPIDI_Datatype * dt); -void -MPIDI_Win_datatype_unmap(MPIDI_Datatype * dt); - -void -MPIDI_WinCtrlSend(pami_context_t context, - MPIDI_Win_control_t *control, - int rank, - MPIR_Win *win); - -void -MPIDI_WinLockReq_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer); -void -MPIDI_WinLockAck_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer); -void -MPIDI_WinUnlock_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer); - -void -MPIDI_WinComplete_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer); -void -MPIDI_WinPost_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer); - -void -MPIDI_Win_DoneCB(pami_context_t context, - void * cookie, - pami_result_t result); -void -MPIDI_WinUnlockDoneCB(pami_context_t context, - void * cookie, - pami_result_t result); - -void -MPIDI_WinAccumCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv); -int -MPIDI_Win_init( MPI_Aint length, - int disp_unit, - MPIR_Win **win_ptr, - MPIR_Info *info, - MPIR_Comm *comm_ptr, - int create_flavor, - int model); -int -MPIDI_Win_allgather(MPI_Aint size, - MPIR_Win **win_ptr); - -void -MPIDI_WinLockAdvance(pami_context_t context, - MPIR_Win * win); -void -MPIDI_WinLockAck_post(pami_context_t context, - unsigned peer, - MPIR_Win *win); - -void -MPIDI_WinLockReq_proc(pami_context_t context, - const MPIDI_Win_control_t * info, - unsigned peer); - -int -MPIDI_Datatype_is_pami_rmw_supported(MPI_Datatype datatype, - pami_type_t *pami_type, - MPI_Op op, - pami_atomic_t *pami_op); - -int -MPIDI_valid_group_rank(int lpid, - MPIR_Group *grp); -void -MPIDI_WinLockAllAck_post(pami_context_t context, - unsigned peer, - MPIR_Win * win); - - -#endif diff --git a/src/mpid/pamid/src/onesided/mpidi_win_control.c b/src/mpid/pamid/src/onesided/mpidi_win_control.c deleted file mode 100644 index f10d45d..0000000 --- a/src/mpid/pamid/src/onesided/mpidi_win_control.c +++ /dev/null @@ -1,149 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/onesided/mpidi_win_control.c - * \brief ??? - */ -#include "mpidi_onesided.h" - - -void -MPIDI_WinCtrlSend(pami_context_t context, - MPIDI_Win_control_t *control, - int rank, - MPIR_Win *win) -{ - pami_task_t taskid; - MPIDI_WinLock_info *winLock; - control->win = win->mpid.info[rank].win; - control->rank = win->comm_ptr->rank; - taskid=MPID_VCR_GET_LPID(win->comm_ptr->vcr,rank); - - pami_endpoint_t dest; - pami_result_t rc; - taskid=MPID_VCR_GET_LPID(win->comm_ptr->vcr,rank); - rc = PAMI_Endpoint_create(MPIDI_Client,taskid, 0, &dest); - MPID_assert(rc == PAMI_SUCCESS); - - if ((control->type == MPIDI_WIN_MSGTYPE_UNLOCK) || - (control->type == MPIDI_WIN_MSGTYPE_UNLOCKALL)) { - pami_send_t params = { - .send = { - .dispatch = MPIDI_Protocols_WinCtrl, - .dest = dest, - .header = { - .iov_base = control, - .iov_len = sizeof(MPIDI_Win_control_t), - }, - }, - .events = { - .cookie = win, - .local_fn = NULL, - .remote_fn= MPIDI_WinUnlockDoneCB, - }, - }; - rc = PAMI_Send(context, ¶ms); - } else { - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_WinCtrl, - .dest = dest, - .header = { - .iov_base = control, - .iov_len = sizeof(MPIDI_Win_control_t), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - }; - rc = PAMI_Send_immediate(context, ¶ms); - } - if ((control->type == MPIDI_WIN_MSGTYPE_LOCKALLREQ) || (control->type == MPIDI_WIN_MSGTYPE_UNLOCKALL)) { - winLock = (MPIDI_WinLock_info *) control->flagAddr; - winLock->done = 1; - } - MPID_assert(rc == PAMI_SUCCESS); - -} - -void -MPIDI_WinUnlockDoneCB(pami_context_t context, - void * cookie, - pami_result_t result) -{ - MPIR_Win *win = (MPIR_Win *)cookie; - if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_LOCK) { - win->mpid.sync.lock.remote.locked = 0; - } - else if (win->mpid.sync.origin_epoch_type == MPID_EPOTYPE_LOCK_ALL) - { - MPID_assert((int)win->mpid.sync.lock.remote.allLocked > 0); - win->mpid.sync.lock.remote.allLocked -= 1; - } - else - MPID_assert_always(0); -} - - -void -MPIDI_WinControlCB(pami_context_t context, - void * cookie, - const void * _control, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv == NULL); - MPID_assert(sndlen == 0); - MPID_assert(_control != NULL); - MPID_assert(size == sizeof(MPIDI_Win_control_t)); - const MPIDI_Win_control_t *control = (const MPIDI_Win_control_t *)_control; - pami_task_t senderrank = PAMIX_Endpoint_query(sender); - - switch (control->type) - { - case MPIDI_WIN_MSGTYPE_LOCKREQ: - case MPIDI_WIN_MSGTYPE_LOCKALLREQ: - MPIDI_WinLockReq_proc(context, control, senderrank); - break; - case MPIDI_WIN_MSGTYPE_LOCKACK: - case MPIDI_WIN_MSGTYPE_LOCKALLACK: - MPIDI_WinLockAck_proc(context, control, senderrank); - break; - case MPIDI_WIN_MSGTYPE_UNLOCK: - case MPIDI_WIN_MSGTYPE_UNLOCKALL: - MPIDI_WinUnlock_proc(context, control, senderrank); - break; - case MPIDI_WIN_MSGTYPE_COMPLETE: - MPIDI_WinComplete_proc(context, control, senderrank); - break; - case MPIDI_WIN_MSGTYPE_POST: - MPIDI_WinPost_proc(context, control, senderrank); - break; - - default: - fprintf(stderr, "Bad win-control type: 0x%08x %d\n", - control->type, - control->type); - MPID_abort(); - } - - MPIDI_Progress_signal(); -} diff --git a/src/mpid/pamid/src/pamix/Makefile.mk b/src/mpid/pamid/src/pamix/Makefile.mk deleted file mode 100644 index 0ab3a81..0000000 --- a/src/mpid/pamid/src/pamix/Makefile.mk +++ /dev/null @@ -1,29 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/pamix/pamix.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/pamix/pamix.c b/src/mpid/pamid/src/pamix/pamix.c deleted file mode 100644 index 4077318..0000000 --- a/src/mpid/pamid/src/pamix/pamix.c +++ /dev/null @@ -1,347 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pamix/pamix.c - * \brief This file contains routines to make the PAMI API usable for MPI internals. It is less likely that - * MPI apps will use these routines. - */ - -#include <assert.h> -#include <limits.h> -#include "mpidimpl.h" -#include <pamix.h> - -#define PAMIX_assert_always(x) assert(x) -#if ASSERT_LEVEL==0 -#define PAMIX_assert(x) -#elif ASSERT_LEVEL>=1 -#define PAMIX_assert(x) assert(x) -#endif - -#ifdef __BGQ__ -#define __BG__ -#include <spi/include/kernel/location.h> -#endif - -typedef pami_result_t (*pamix_progress_register_fn) (pami_context_t context, - pamix_progress_function progress_fn, - pamix_progress_function suspend_fn, - pamix_progress_function resume_fn, - void * cookie); -typedef pami_result_t (*pamix_progress_enable_fn) (pami_context_t context, - pamix_progress_t event_type); -typedef pami_result_t (*pamix_progress_disable_fn) (pami_context_t context, - pamix_progress_t event_type); - -#if defined(__BG__) -typedef const pamix_torus_info_t* (*pamix_torus_info_fn) (); -typedef pami_result_t (*pamix_task2torus_fn) (pami_task_t, size_t[]); -typedef pami_result_t (*pamix_torus2task_fn) (size_t[], pami_task_t *); -#endif - - -static struct -{ - pamix_progress_register_fn progress_register; - pamix_progress_enable_fn progress_enable; - pamix_progress_disable_fn progress_disable; - -#if defined(__BG__) - pamix_torus_info_fn torus_info; - pamix_task2torus_fn task2torus; - pamix_torus2task_fn torus2task; -#endif -} PAMIX_Functions = {0}; - -pamix_extension_info_t PAMIX_Extensions; - - -#define PAMI_EXTENSION_OPEN(client, name, ext) \ -({ \ - pami_result_t rc; \ - rc = PAMI_Extension_open(client, name, ext); \ - PAMIX_assert_always(rc == PAMI_SUCCESS); \ -}) -#define PAMI_EXTENSION_FUNCTION(type, name, ext) \ -({ \ - void* fn; \ - fn = PAMI_Extension_symbol(ext, name); \ - (type)fn; \ -}) -void -PAMIX_Initialize(pami_client_t client) -{ - PAMI_EXTENSION_OPEN(client, "EXT_async_progress", &PAMIX_Extensions.progress); - PAMIX_Functions.progress_register = PAMI_EXTENSION_FUNCTION(pamix_progress_register_fn, "register", PAMIX_Extensions.progress); - PAMIX_Functions.progress_enable = PAMI_EXTENSION_FUNCTION(pamix_progress_enable_fn, "enable", PAMIX_Extensions.progress); - PAMIX_Functions.progress_disable = PAMI_EXTENSION_FUNCTION(pamix_progress_disable_fn, "disable", PAMIX_Extensions.progress); - -#if defined(__BG__) - PAMI_EXTENSION_OPEN(client, "EXT_torus_network", &PAMIX_Extensions.torus); - PAMIX_Functions.torus_info = PAMI_EXTENSION_FUNCTION(pamix_torus_info_fn, "information", PAMIX_Extensions.torus); - PAMIX_Functions.task2torus = PAMI_EXTENSION_FUNCTION(pamix_task2torus_fn, "task2torus", PAMIX_Extensions.torus); - PAMIX_Functions.torus2task = PAMI_EXTENSION_FUNCTION(pamix_torus2task_fn, "torus2task", PAMIX_Extensions.torus); -#endif - -#ifdef PAMIX_IS_LOCAL_TASK - { - PAMIX_Extensions.is_local_task.base = NULL; - PAMIX_Extensions.is_local_task.stride = 0; - PAMIX_Extensions.is_local_task.bitmask = 0; - PAMIX_Extensions.is_local_task.node_info = NULL; - PAMIX_Extensions.is_local_task.status = PAMI_Extension_open(client, "EXT_is_local_task", - &PAMIX_Extensions.is_local_task.extension); - - if (PAMIX_Extensions.is_local_task.status == PAMI_SUCCESS) - { - PAMIX_Extensions.is_local_task.base = PAMI_EXTENSION_FUNCTION(uint8_t *, "base", PAMIX_Extensions.is_local_task.extension); - PAMIX_Extensions.is_local_task.stride = PAMI_EXTENSION_FUNCTION(uintptr_t, "stride", PAMIX_Extensions.is_local_task.extension); - PAMIX_Extensions.is_local_task.bitmask = PAMI_EXTENSION_FUNCTION(uintptr_t, "bitmask", PAMIX_Extensions.is_local_task.extension); -#if defined(MPID_USE_NODE_IDS) - PAMIX_Extensions.is_local_task.node_info = PAMI_EXTENSION_FUNCTION(node_info_fn, "get_node_info", PAMIX_Extensions.is_local_task.extension); - if (PAMIX_Extensions.is_local_task.node_info == NULL) - { - /* The "node information" is not available via the "is_local_task" - * extension to pami - possibly due to a downlevel version of pami. - * */ - PAMIX_Extensions.is_local_task.node_info = (node_info_fn) PAMIX_is_local_task_get_node_info; - } -#endif - } - -#if defined(PAMIX_IS_LOCAL_TASK_STRIDE) && defined(PAMIX_IS_LOCAL_TASK_BITMASK) - /* - * If the compile-time stride and bitmask values are defined, then assert - * that the extension was open successfully and the base pointer is valid to - * avoid a null dereference in the PAMIX_Task_is_local macro and assert that - * the compile-time values match the values specified by the extension. - */ - PAMIX_assert_always(PAMIX_Extensions.is_local_task.status == PAMI_SUCCESS); - PAMIX_assert_always(PAMIX_Extensions.is_local_task.base != NULL); - PAMIX_assert_always(PAMIX_IS_LOCAL_TASK_STRIDE == PAMIX_Extensions.is_local_task.stride); - PAMIX_assert_always(PAMIX_IS_LOCAL_TASK_BITMASK == PAMIX_Extensions.is_local_task.bitmask); -#endif - } -#endif /* PAMIX_IS_LOCAL_TASK */ -} - - -void -PAMIX_Finalize(pami_client_t client) -{ - pami_result_t rc; - rc = PAMI_Extension_close(PAMIX_Extensions.progress); - PAMIX_assert_always(rc == PAMI_SUCCESS); - - if (PAMIX_Extensions.is_local_task.status == PAMI_SUCCESS) - { - rc = PAMI_Extension_close(PAMIX_Extensions.is_local_task.extension); - PAMIX_assert_always(rc == PAMI_SUCCESS); - - PAMIX_Extensions.is_local_task.base = NULL; - PAMIX_Extensions.is_local_task.stride = 0; - PAMIX_Extensions.is_local_task.bitmask = 0; - PAMIX_Extensions.is_local_task.status = PAMI_ERROR; - } - -#if defined(__BG__) - rc = PAMI_Extension_close(PAMIX_Extensions.torus); - PAMIX_assert_always(rc == PAMI_SUCCESS); -#endif -} - - -pami_configuration_t -PAMIX_Client_query(pami_client_t client, - pami_attribute_name_t name) -{ - pami_result_t rc; - pami_configuration_t query; - query.name = name; - rc = PAMI_Client_query(client, &query, 1); - PAMIX_assert_always(rc == PAMI_SUCCESS); - return query; -} - - -static inline pami_configuration_t -PAMIX_Dispatch_query(pami_context_t context, - size_t dispatch, - pami_attribute_name_t name) -{ - pami_result_t rc; - pami_configuration_t query; - query.name = name; - rc = PAMI_Dispatch_query(context, dispatch, &query, 1); - PAMIX_assert_always(rc == PAMI_SUCCESS); - return query; -} - - -void -PAMIX_Dispatch_set(pami_context_t context[], - size_t num_contexts, - size_t dispatch, - pami_dispatch_callback_function fn, - pami_dispatch_hint_t options, - size_t * immediate_max) -{ - pami_result_t rc; - size_t i; - size_t last_immediate_max = (size_t)-1; - for (i=0; i<num_contexts; ++i) { - rc = PAMI_Dispatch_set(context[i], - dispatch, - fn, - (void*)i, - options); - PAMIX_assert_always(rc == PAMI_SUCCESS); - - size_t size; - size = PAMIX_Dispatch_query(context[i], dispatch, PAMI_DISPATCH_SEND_IMMEDIATE_MAX).value.intval; - last_immediate_max = MPL_MIN(size, last_immediate_max); - size = PAMIX_Dispatch_query(context[i], dispatch, PAMI_DISPATCH_RECV_IMMEDIATE_MAX).value.intval; - last_immediate_max = MPL_MIN(size, last_immediate_max); - } - - if (immediate_max != NULL) - *immediate_max = last_immediate_max; -} - - -pami_task_t -PAMIX_Endpoint_query(pami_endpoint_t endpoint) -{ - pami_task_t rank; - size_t offset; - - pami_result_t rc; - rc = PAMI_Endpoint_query(endpoint, &rank, &offset); - PAMIX_assert(rc == PAMI_SUCCESS); - - return rank; -} - - -void -PAMIX_Progress_register(pami_context_t context, - pamix_progress_function progress_fn, - pamix_progress_function suspend_fn, - pamix_progress_function resume_fn, - void * cookie) -{ - PAMIX_assert_always(PAMIX_Functions.progress_register != NULL); - pami_result_t rc; - rc = PAMIX_Functions.progress_register(context, progress_fn,suspend_fn, resume_fn, cookie); - PAMIX_assert_always(rc == PAMI_SUCCESS); -} - - -void -PAMIX_Progress_enable(pami_context_t context, - pamix_progress_t event_type) -{ - PAMIX_assert(PAMIX_Functions.progress_enable != NULL); - pami_result_t rc; - rc = PAMIX_Functions.progress_enable(context, event_type); - PAMIX_assert(rc == PAMI_SUCCESS); -} - - -void -PAMIX_Progress_disable(pami_context_t context, - pamix_progress_t event_type) -{ - PAMIX_assert(PAMIX_Functions.progress_disable != NULL); - pami_result_t rc; - rc = PAMIX_Functions.progress_disable(context, event_type); - PAMIX_assert(rc == PAMI_SUCCESS); -} - - -pami_result_t -PAMIX_is_local_task_get_node_info(pami_task_t task, - uint32_t *node_id, - uint32_t *offset, - uint32_t *max_nodes) -{ -#if defined(__BG__) - size_t coords[5]; - if (PAMI_SUCCESS == PAMIX_Task2torus(task, coords)) - { - const pamix_torus_info_t * info = PAMIX_Torus_info(); - - *node_id = - coords[0] + - coords[1] * info->size[0] + - coords[2] * info->size[0] * info->size[1] + - coords[3] * info->size[0] * info->size[1] * info->size[2] + - coords[4] * info->size[0] * info->size[1] * info->size[2] * info->size[3]; - - *max_nodes = info->size[0] * info->size[1] * info->size[2] * info->size[3] * info->size[4]; - - *offset = 0; /* ???????????????????????????????? */ - } -#else - MPID_abort(); /* Other platforms should not need a fallback implementation */ -#endif - - return PAMI_SUCCESS; -} - -#if defined(__BG__) - -const pamix_torus_info_t * -PAMIX_Torus_info() -{ - PAMIX_assert(PAMIX_Functions.torus_info != NULL); - return PAMIX_Functions.torus_info(); -} - - -int -PAMIX_Task2torus(pami_task_t task_id, - size_t coords[]) -{ - PAMIX_assert(PAMIX_Functions.task2torus != NULL); - pami_result_t rc; - rc = PAMIX_Functions.task2torus(task_id, coords); - return rc; -} - - -#include <stdio.h> -int -PAMIX_Torus2task(size_t coords[], - pami_task_t * task_id) -{ - PAMIX_assert(PAMIX_Functions.torus2task != NULL); - pami_result_t rc; - rc = PAMIX_Functions.torus2task(coords, task_id); - return rc; -#if 0 - if(rc != PAMI_SUCCESS) - { - fprintf(stderr,"coords in: %zu %zu %zu %zu %zu, rc: %d\n", coords[0], coords[1], coords[2],coords[3],coords[4], rc); - PAMIX_assert(rc == PAMI_SUCCESS); - } - else return rc; -#endif -} - -#endif diff --git a/src/mpid/pamid/src/pt2pt/Makefile.mk b/src/mpid/pamid/src/pt2pt/Makefile.mk deleted file mode 100644 index c8dc0e4..0000000 --- a/src/mpid/pamid/src/pt2pt/Makefile.mk +++ /dev/null @@ -1,50 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -noinst_HEADERS += \ - src/mpid/pamid/src/pt2pt/mpid_send.h \ - src/mpid/pamid/src/pt2pt/mpidi_recv.h \ - src/mpid/pamid/src/pt2pt/mpidi_send.h \ - src/mpid/pamid/src/pt2pt/mpid_irecv.h \ - src/mpid/pamid/src/pt2pt/mpid_isend.h - -include $(top_srcdir)/src/mpid/pamid/src/pt2pt/persistent/Makefile.mk - -mpi_core_sources += \ - src/mpid/pamid/src/pt2pt/mpid_cancel.c \ - src/mpid/pamid/src/pt2pt/mpid_issend.c \ - src/mpid/pamid/src/pt2pt/mpid_recv.c \ - src/mpid/pamid/src/pt2pt/mpid_ssend.c \ - src/mpid/pamid/src/pt2pt/mpidi_callback_eager.c \ - src/mpid/pamid/src/pt2pt/mpidi_callback_rzv.c \ - src/mpid/pamid/src/pt2pt/mpidi_callback_short.c \ - src/mpid/pamid/src/pt2pt/mpidi_callback_util.c \ - src/mpid/pamid/src/pt2pt/mpidi_control.c \ - src/mpid/pamid/src/pt2pt/mpidi_done.c \ - src/mpid/pamid/src/pt2pt/mpidi_recvmsg.c \ - src/mpid/pamid/src/pt2pt/mpidi_rendezvous.c \ - src/mpid/pamid/src/pt2pt/mpidi_sendmsg.c - - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/pt2pt/mpid_cancel.c b/src/mpid/pamid/src/pt2pt/mpid_cancel.c deleted file mode 100644 index 9b8d30f..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_cancel.c +++ /dev/null @@ -1,138 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_cancel.c - * \brief Device interface for canceling an MPI Recv - */ -#include <mpidimpl.h> - - -int -MPID_Cancel_recv(MPIR_Request * rreq) -{ - MPID_assert(rreq->kind == MPIR_REQUEST_KIND__RECV); - if (MPIDI_Recvq_FDPR(rreq)) - { - MPIR_STATUS_SET_CANCEL_BIT(rreq->status, TRUE); - MPIR_STATUS_SET_COUNT(rreq->status, 0); - MPIDI_Request_complete(rreq); - } - /* This is successful, even if the recv isn't cancelled */ - return MPI_SUCCESS; -} - - -/** - * \brief Cancel an MPI_Isend() - * - * \param[in] req The request element to cancel - * - * \return The same as MPIDI_CtrlSend() - */ -static inline pami_result_t -MPIDI_CancelReq_post(pami_context_t context, void * _req) -{ - MPIR_Request * req = (MPIR_Request*)_req; - MPID_assert(req != NULL); - - /* ------------------------------------------------- */ - /* Check if we already have a cancel request pending */ - /* ------------------------------------------------- */ - if (MPIDI_Request_cancel_pending(req)) - { - MPIDI_Request_complete(req); - return PAMI_SUCCESS; - } - - MPIDI_MsgInfo cancel = { - .MPItag = MPIDI_Request_getMatchTag(req), - .MPIrank = MPIDI_Request_getMatchRank(req), - .MPIctxt = MPIDI_Request_getMatchCtxt(req), - .req = req->handle, - }; - cancel.control = MPIDI_CONTROL_CANCEL_REQUEST; - - pami_endpoint_t dest; - MPIDI_Context_endpoint(req, &dest); - - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_Cancel, - .dest = dest, - .header = { - .iov_base= &cancel, - .iov_len= sizeof(MPIDI_MsgInfo), - }, - }; - - TRACE_ERR("Running posted cancel for request=%p local=%u remote=%u\n", - req, MPIR_Process.comm_world->rank, MPIDI_Request_getPeerRank_pami(req)); - pami_result_t rc; - rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - - return PAMI_SUCCESS; -} - - -int -MPID_Cancel_send(MPIR_Request * sreq) -{ - MPID_assert(sreq != NULL); - - if(!sreq->comm) - return MPI_SUCCESS; - - MPIDI_Request_uncomplete(sreq); - /* TRACE_ERR("Posting cancel for request=%p cc(curr)=%d ref(curr)=%d\n", sreq, val+1, MPIR_Object_get_ref(sreq)); */ - - pami_context_t context = MPIDI_Context_local(sreq); - -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - if (likely(MPIDI_Process.perobj.context_post.active > 0)) - { - /* This leaks intentionally. At this time, the amount of work - * required to avoid a leak here just isn't worth it. - * Hopefully people aren't cancelling sends too much. - */ - pami_work_t * work = MPL_malloc(sizeof(pami_work_t)); - PAMI_Context_post(context, work, MPIDI_CancelReq_post, sreq); - } - else - { - /* Lock access to the context. For more information see discussion of the - * simplifying assumption that the "per object" mpich lock mode does not - * expect a completely single threaded run environment in the file - * src/mpid/pamid/src/mpid_progress.h - */ - PAMI_Context_lock(context); - MPIDI_CancelReq_post(context, sreq); - PAMI_Context_unlock(context); - } -#else /* (MPICH_THREAD_GRANULARITY != MPICH_THREAD_GRANULARITY__POBJ) */ - /* - * It is not necessary to lock the context before access in the "global" mpich - * lock mode because all application threads must first acquire the global - * mpich lock upon entry into the library, and any active async threads must - * first acquire the global mpich lock before accessing internal mpich data - * structures or accessing pami objects. This makes the context lock redundant. - */ - MPIDI_CancelReq_post(context, sreq); -#endif - - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/pt2pt/mpid_irecv.h b/src/mpid/pamid/src/pt2pt/mpid_irecv.h deleted file mode 100644 index a0b6daf..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_irecv.h +++ /dev/null @@ -1,67 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_irecv.h - * \brief ADI level implemenation of MPI_Irecv() - */ -#ifndef __src_pt2pt_mpid_irecv_h__ -#define __src_pt2pt_mpid_irecv_h__ - -#include <mpidimpl.h> -#include "mpidi_recv.h" - -#define MPID_Irecv MPID_Irecv_inline - -/** - * \brief ADI level implemenation of MPI_Irecv() - * - * \param[in] buf The buffer to receive into - * \param[in] count Number of expected elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The sending rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - */ -static inline int -MPID_Irecv_inline(void * buf, - MPI_Aint count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - return MPIDI_Recv(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - 0, - MPI_STATUS_IGNORE, - request); -} - -#endif diff --git a/src/mpid/pamid/src/pt2pt/mpid_isend.h b/src/mpid/pamid/src/pt2pt/mpid_isend.h deleted file mode 100644 index 7928908..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_isend.h +++ /dev/null @@ -1,129 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_isend.h - * \brief ??? - */ - -#ifndef __src_pt2pt_mpid_isend_h__ -#define __src_pt2pt_mpid_isend_h__ - - -#define MPID_Isend MPID_Isend_inline - - -static inline unsigned -MPIDI_Context_hash(pami_task_t rank, unsigned ctxt, unsigned bias, unsigned ncontexts) -{ -#if (MPICH_THREAD_GRANULARITY == MPICH_THREAD_GRANULARITY__POBJ) - return (( rank + ctxt + bias ) & (ncontexts-1)); -#else - /* The 'global' mpich lock mode only supports a single context. See - * discussion in mpich/src/mpid/pamid/src/mpid_init.c for more information. - */ - return 0; -#endif -} -static inline void -MPIDI_Context_endpoint(MPIR_Request * req, pami_endpoint_t * e) -{ - pami_task_t remote = MPIDI_Request_getPeerRank_pami(req); - pami_task_t local = MPIR_Process.comm_world->rank; - unsigned rctxt = MPIDI_Context_hash(local, req->comm->context_id, MPIDI_Process.avail_contexts>>1, MPIDI_Process.avail_contexts); - - pami_result_t rc; - rc = PAMI_Endpoint_create(MPIDI_Client, remote, rctxt, e); - MPID_assert(rc == PAMI_SUCCESS); -} -static inline pami_context_t -MPIDI_Context_local(MPIR_Request * req) -{ - pami_task_t remote = MPIDI_Request_getPeerRank_comm(req); - unsigned lctxt = MPIDI_Context_hash(remote, req->comm->context_id, 0, MPIDI_Process.avail_contexts); - MPID_assert(lctxt < MPIDI_Process.avail_contexts); - return MPIDI_Context[lctxt]; -} - - -/** - * \brief ADI level implemenation of MPI_Isend() - * - * \param[in] buf The buffer to send - * \param[in] count Number of elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The destination rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - * - * This is a slight variation on mpid_send.c - basically, we *always* - * want to return a send request even if the request is already - * complete (as is in the case of sending to a NULL rank). - */ -static inline int -MPID_Isend_inline(const void * buf, - MPI_Aint count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - /* ---------------------------------------------------- */ - /* special case: PROC null handled by handoff function */ - /* ---------------------------------------------------- */ - - /* --------------------- */ - /* create a send request */ - /* --------------------- */ - MPIR_Request * sreq = *request = MPIDI_Request_create2_fast(); - - unsigned context_id = comm->context_id; - /* match info */ - MPIDI_Request_setMatch(sreq, tag, comm->rank, context_id+context_offset); - - /* data buffer info */ - sreq->mpid.userbuf = (void*)buf; - sreq->mpid.userbufcount = count; - sreq->mpid.datatype = datatype; - - /* Enable passing in MPI_PROC_NULL, do the translation in the - handoff function */ - MPIDI_Request_setPeerRank_comm(sreq, rank); - - unsigned ncontexts = MPIDI_Process.avail_contexts; - /* communicator & destination info */ - sreq->comm = comm; - sreq->kind = MPIR_REQUEST_KIND__SEND; - MPIR_Comm_add_ref(comm); - - pami_context_t context = MPIDI_Context[MPIDI_Context_hash(rank, context_id, 0, ncontexts)]; - if (context_offset == 0) - MPIDI_Context_post(context, &sreq->mpid.post_request, MPIDI_Isend_handoff, sreq); - else - MPIDI_Context_post(context, &sreq->mpid.post_request, MPIDI_Isend_handoff_internal, sreq); - - return MPI_SUCCESS; -} - - -#endif diff --git a/src/mpid/pamid/src/pt2pt/mpid_issend.c b/src/mpid/pamid/src/pt2pt/mpid_issend.c deleted file mode 100644 index 21510a5..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_issend.c +++ /dev/null @@ -1,62 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_issend.c - * \brief ADI level implemenation of MPI_Issend() - */ -#include <mpidimpl.h> -#include "mpidi_send.h" - -/** - * \brief ADI level implemenation of MPI_Issend() - * - * \param[in] buf The buffer to send - * \param[in] count Number of elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The destination rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - * - * This is a slight variation on mpid_ssend.c - basically, we *always* - * want to return a send request even if the request is already - * complete (as is in the case of sending to a NULL rank). - */ -int MPID_Issend(const void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - return MPIDI_Send(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - 0, - 1, - request); -} diff --git a/src/mpid/pamid/src/pt2pt/mpid_recv.c b/src/mpid/pamid/src/pt2pt/mpid_recv.c deleted file mode 100644 index 15c88b9..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_recv.c +++ /dev/null @@ -1,60 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_recv.c - * \brief ADI level implemenation of MPI_Recv() - */ -#include <mpidimpl.h> -#include "mpidi_recv.h" - -/** - * \brief ADI level implemenation of MPI_Recv() - * - * \param[in] buf The buffer to receive into - * \param[in] count Number of expected elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The sending rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[out] status Update the status structure - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - */ -int MPID_Recv(void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPI_Status * status, - MPIR_Request ** request) -{ - return MPIDI_Recv(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - 0, - status, - request); -} diff --git a/src/mpid/pamid/src/pt2pt/mpid_send.h b/src/mpid/pamid/src/pt2pt/mpid_send.h deleted file mode 100644 index 86c23f1..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_send.h +++ /dev/null @@ -1,67 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_send.h - * \brief ADI level implemenation of MPI_Send() - */ -#ifndef __src_pt2pt_mpid_send_h__ -#define __src_pt2pt_mpid_send_h__ - -#include <mpidimpl.h> -#include "mpidi_send.h" - -#define MPID_Send MPID_Send_inline - -/** - * \brief ADI level implemenation of MPI_Send() - * - * \param[in] buf The buffer to send - * \param[in] count Number of elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The destination rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - */ -static inline int -MPID_Send_inline(const void * buf, - MPI_Aint count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - return MPIDI_Send(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - 1, - 0, - request); -} - - -#endif diff --git a/src/mpid/pamid/src/pt2pt/mpid_ssend.c b/src/mpid/pamid/src/pt2pt/mpid_ssend.c deleted file mode 100644 index 58ce5f9..0000000 --- a/src/mpid/pamid/src/pt2pt/mpid_ssend.c +++ /dev/null @@ -1,58 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpid_ssend.c - * \brief ADI level implemenation of MPI_Ssend() - */ -#include <mpidimpl.h> -#include "mpidi_send.h" - -/** - * \brief ADI level implemenation of MPI_Ssend() - * - * \param[in] buf The buffer to send - * \param[in] count Number of elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The destination rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - */ -int MPID_Ssend(const void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - return MPIDI_Send(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - 1, - 1, - request); -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_callback_eager.c b/src/mpid/pamid/src/pt2pt/mpidi_callback_eager.c deleted file mode 100644 index 51b876e..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_callback_eager.c +++ /dev/null @@ -1,312 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_callback_eager.c - * \brief The callback for a new eager message - */ -#include <mpidimpl.h> - - -/** - * \brief The standard callback for a new message - * - * \param[in] context The context on which the message is being received. - * \param[in] cookie Unused - * \param[in] _msginfo The header information - * \param[in] msginfo_size The size of the header information - * \param[in] sndbuf If the message is short, this is the data - * \param[in] sndlen The size of the incoming data - * \param[out] recv If the message is long, this tells the message layer how to handle the data. - */ -void -MPIDI_RecvCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - const MPIDI_MsgInfo *msginfo = (const MPIDI_MsgInfo *)_msginfo; -#if TOKEN_FLOW_CONTROL - int rettoks=0; - void *uebuf; -#endif - pami_task_t source; - if (recv == NULL) - { - if (msginfo->isSync) - MPIDI_RecvShortSyncCB(context, - cookie, - _msginfo, - msginfo_size, - sndbuf, - sndlen, - sender, - recv); - else - MPIDI_RecvShortAsyncCB(context, - cookie, - _msginfo, - msginfo_size, - sndbuf, - sndlen, - sender, - recv); - return; - } - - MPID_assert(sndbuf == NULL); - MPID_assert(_msginfo != NULL); - MPID_assert(msginfo_size == sizeof(MPIDI_MsgInfo)); - - MPIR_Request * rreq = NULL; - source=PAMIX_Endpoint_query(sender); - - /* -------------------- */ - /* Match the request. */ - /* -------------------- */ - unsigned rank = msginfo->MPIrank; - unsigned tag = msginfo->MPItag; - unsigned context_id = msginfo->MPIctxt; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - MPIDI_Receive_tokens(msginfo,source); - #else - MPID_assert_always(0); - #endif - } -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDP(rank, tag, context_id); -#else - rreq = MPIDI_Recvq_FDP(rank, source, tag, context_id, msginfo->MPIseqno); -#endif - - /* Match not found */ - if (unlikely(rreq == NULL)) - { - /* No request was found and hence no sync needed */ -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivals); -#endif - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIR_Request *newreq = MPIDI_Request_create2(); - MPID_assert(newreq != NULL); - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - #else - MPID_assert_always(0); - #endif - } - - if (sndlen) - { - newreq->mpid.uebuflen = sndlen; - if (!(TOKEN_FLOW_CONTROL_ON)) - { - newreq->mpid.uebuf = MPL_malloc(sndlen); - newreq->mpid.uebuf_malloc = mpiuMalloc ; - } - else - { - #if TOKEN_FLOW_CONTROL - newreq->mpid.uebuf = MPIDI_mm_alloc(sndlen); - newreq->mpid.uebuf_malloc = mpidiBufMM; - #else - MPID_assert_always(0); - #endif - } - MPID_assert(newreq->mpid.uebuf != NULL); - } - if (!TOKEN_FLOW_CONTROL_ON) - { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - } -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDP(rank, tag, context_id); -#else - rreq = MPIDI_Recvq_FDP(rank, source, tag, context_id, msginfo->MPIseqno); -#endif - - if (unlikely(rreq == NULL)) - { - MPIDI_Callback_process_unexp(newreq, context, msginfo, sndlen, sender, sndbuf, recv, msginfo->isSync); - int completed = MPIR_Request_is_complete(newreq); - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - MPIDI_Token_cntr[sender].unmatched++; - #else - MPID_assert_always(0); - #endif - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - if (completed) MPIR_Request_free(newreq); - goto fn_exit_eager; - } - else - { - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPID_Request_discard(newreq); - } - } - else - { -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivalsMatched); -#endif - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - MPIDI_Update_rettoks(source); - MPIDI_Must_return_tokens(context,source); - #else - MPID_assert_always(0); - #endif - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } - - /* -------------------------------------------- */ - /* Figure out target buffer for request data. */ - /* -------------------------------------------- */ - - /* ---------------------- */ - /* Copy in information. */ - /* ---------------------- */ - rreq->status.MPI_SOURCE = rank; - rreq->status.MPI_TAG = tag; - MPIR_STATUS_SET_COUNT(rreq->status, sndlen); - MPIDI_Request_setCA (rreq, MPIDI_CA_COMPLETE); - MPIDI_Request_cpyPeerRequestH(rreq, msginfo); - MPIDI_Request_setSync (rreq, msginfo->isSync); - MPIDI_Request_setRzv (rreq, 0); - TRACE_SET_R_VAL(source,(rreq->mpid.PR_idx),len,sndlen); - /* --------------------------------------- */ - /* We have to fill in the callback info. */ - /* --------------------------------------- */ - recv->local_fn = MPIDI_RecvDoneCB_mutexed; - recv->cookie = rreq; -#if ASSERT_LEVEL > 0 - /* This ensures that the value is set, even to something impossible */ - recv->addr = NULL; -#endif - - /* ----------------------------- */ - /* Request was already posted. */ - /* ----------------------------- */ - - if (unlikely(msginfo->isSync)) - MPIDI_SyncAck_post(context, rreq, source); - - /* ----------------------------------------- */ - /* Calculate message length for reception. */ - /* ----------------------------------------- */ - unsigned dt_contig, dt_size; - MPIDU_Datatype*dt_ptr; - MPI_Aint dt_true_lb; - MPIDI_Datatype_get_info(rreq->mpid.userbufcount, - rreq->mpid.datatype, - dt_contig, - dt_size, - dt_ptr, - dt_true_lb); - - /* ----------------------------- */ - /* Test for truncated message. */ - /* ----------------------------- */ - if (unlikely(sndlen > dt_size)) - { - MPIDI_Callback_process_trunc(context, rreq, recv, sndbuf); - goto fn_exit_eager; - } - - /* --------------------------------------- */ - /* If buffer is contiguous, we are done. */ - /* --------------------------------------- */ - if (likely(dt_contig)) - { - /* - * This is to test that the fields don't need to be - * initialized. Remove after this doesn't fail for a while. - */ - MPID_assert(rreq->mpid.uebuf == NULL); - MPID_assert(rreq->mpid.uebuflen == 0); - /* rreq->mpid.uebuf = NULL; */ - /* rreq->mpid.uebuflen = 0; */ - void* rcvbuf = rreq->mpid.userbuf + dt_true_lb; - - recv->addr = rcvbuf; - } - - /* ----------------------------------------------- */ - /* Buffer is non-contiguous. we need to allocate */ - /* a temporary buffer, and unpack later. */ - /* ----------------------------------------------- */ - else - { - /* ----------------------------------------------- */ - /* Buffer is non-contiguous. the data is already */ - /* available, so we can just unpack it now. */ - /* ----------------------------------------------- */ - MPIDI_Request_setCA(rreq, MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE); - rreq->mpid.uebuflen = sndlen; - if (sndlen) - { - if (!TOKEN_FLOW_CONTROL_ON) - { - rreq->mpid.uebuf = MPL_malloc(sndlen); - rreq->mpid.uebuf_malloc = mpiuMalloc; - } - else - { - #if TOKEN_FLOW_CONTROL - MPIDI_Alloc_lock(&rreq->mpid.uebuf,sndlen); - rreq->mpid.uebuf_malloc = mpidiBufMM; - #else - MPID_assert_always(0); - #endif - } - MPID_assert(rreq->mpid.uebuf != NULL); - } - /* -------------------------------------------------- */ - /* Let PAMI know where to put the rest of the data. */ - /* -------------------------------------------------- */ - recv->addr = rreq->mpid.uebuf; - } - TRACE_SET_R_VAL(source,(rreq->mpid.idx),fl.f.comp_in_HH,2); - TRACE_SET_R_VAL(source,(rreq->mpid.idx),bufadd,rreq->mpid.userbuf); - - fn_exit_eager: -#ifdef OUT_OF_ORDER_HANDLING - if (MPIDI_In_cntr[source].n_OutOfOrderMsgs > 0) { - MPIDI_Recvq_process_out_of_order_msgs(source, context); - } -#endif - - /* ---------------------------------------- */ - /* Signal that the recv has been started. */ - /* ---------------------------------------- */ - MPIDI_Progress_signal(); -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_callback_rzv.c b/src/mpid/pamid/src/pt2pt/mpidi_callback_rzv.c deleted file mode 100644 index 40ae04c..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_callback_rzv.c +++ /dev/null @@ -1,234 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_callback_rzv.c - * \brief The callback for a new RZV RTS - */ -#include <mpidimpl.h> - - -/** - * \brief The callback for a new RZV RTS - * \note Because this is a short message, the data is already received - * \param[in] context The context on which the message is being received. - * \param[in] sender The origin endpoint - * \param[in] _msginfo The extended header information - * \param[in] msginfo_size The size of the extended header information - * \param[in] is_zero_byte The rendezvous message is zero bytes in length. - */ -void -MPIDI_RecvRzvCB_impl(pami_context_t context, - pami_endpoint_t sender, - const void * _msginfo, - size_t msginfo_size, - const unsigned is_zero_byte) -{ - MPID_assert(_msginfo != NULL); - MPID_assert(msginfo_size == sizeof(MPIDI_MsgEnvelope)); - const MPIDI_MsgEnvelope * envelope = (const MPIDI_MsgEnvelope *)_msginfo; - const MPIDI_MsgInfo * msginfo = (const MPIDI_MsgInfo *)&envelope->msginfo; - - MPIR_Request * rreq = NULL; - int found; - pami_task_t source; -#if TOKEN_FLOW_CONTROL - int rettoks=0; -#endif - - /* -------------------- */ - /* Match the request. */ - /* -------------------- */ - unsigned rank = msginfo->MPIrank; - unsigned tag = msginfo->MPItag; - unsigned context_id = msginfo->MPIctxt; - - MPIR_Request *newreq = MPIDI_Request_create2(); - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - source = PAMIX_Endpoint_query(sender); - MPIDI_Receive_tokens(msginfo,source); -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDP_or_AEU(newreq, rank, tag, context_id, &found); -#else - rreq = MPIDI_Recvq_FDP_or_AEU(newreq, rank, source, tag, context_id, msginfo->MPIseqno, &found); -#endif - TRACE_ERR("RZV CB for req=%p remote-mr=0x%llx bytes=%zu (%sfound)\n", - rreq, - *(unsigned long long*)&envelope->envelope.memregion, - envelope->envelope.length, - found?"":"not "); - - /* ---------------------- */ - /* Copy in information. */ - /* ---------------------- */ - rreq->status.MPI_SOURCE = rank; - rreq->status.MPI_TAG = tag; - MPIR_STATUS_SET_COUNT(rreq->status, envelope->length); - MPIDI_Request_setPeerRank_comm(rreq, rank); - MPIDI_Request_setPeerRank_pami(rreq, source); - MPIDI_Request_cpyPeerRequestH (rreq, msginfo); - MPIDI_Request_setSync (rreq, msginfo->isSync); - MPIDI_Request_setRzv (rreq, 1); - - /* ----------------------------------------------------- */ - /* Save the rendezvous information for when the target */ - /* node calls a receive function and the data is */ - /* retreived from the origin node. */ - /* ----------------------------------------------------- */ - if (is_zero_byte) - { - rreq->mpid.envelope.length = 0; - rreq->mpid.envelope.data = NULL; - } - else - { -#ifdef USE_PAMI_RDMA - memcpy(&rreq->mpid.envelope.memregion, - &envelope->memregion, - sizeof(pami_memregion_t)); -#else - rreq->mpid.envelope.memregion_used = envelope->memregion_used; - if(envelope->memregion_used) - { - memcpy(&rreq->mpid.envelope.memregion, - &envelope->memregion, - sizeof(pami_memregion_t)); - } - rreq->mpid.envelope.data = envelope->data; -#endif - rreq->mpid.envelope.length = envelope->length; - TRACE_SET_R_VAL(source,(rreq->mpid.idx),req,rreq); - TRACE_SET_R_VAL(source,(rreq->mpid.idx),rlen,envelope->length); - TRACE_SET_R_VAL(source,(rreq->mpid.idx),fl.f.sync,msginfo->isSync); - TRACE_SET_R_BIT(source,(rreq->mpid.idx),fl.f.rzv); - if (TOKEN_FLOW_CONTROL_ON) - { - #if TOKEN_FLOW_CONTROL - MPIDI_Must_return_tokens(context,source); - #else - MPID_assert_always(0); - #endif - } - } - /* ----------------------------------------- */ - /* figure out target buffer for request data */ - /* ----------------------------------------- */ - if (found) - { -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivalsMatched); -#endif - /* --------------------------- */ - /* if synchronized, post ack. */ - /* --------------------------- */ - if (unlikely(MPIDI_Request_isSync(rreq))) - MPIDI_SyncAck_post(context, rreq, MPIDI_Request_getPeerRank_pami(rreq)); - - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - - if (is_zero_byte) - MPIDI_RecvRzvDoneCB_zerobyte(context, rreq, PAMI_SUCCESS); - else - { - MPIDI_RendezvousTransfer(context, rreq); - TRACE_SET_R_BIT(source,(rreq->mpid.idx),fl.f.sync_com_in_HH); - TRACE_SET_R_BIT(source,(rreq->mpid.idx),fl.f.matchedInHH); - TRACE_SET_R_VAL(source,(rreq->mpid.idx),bufadd,rreq->mpid.userbuf); - } - MPID_Request_discard(newreq); - } - - /* ------------------------------------------------------------- */ - /* Request was not posted. */ - /* ------------------------------------------------------------- */ - else - { -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivals); -#endif - /* - * This is to test that the fields don't need to be - * initialized. Remove after this doesn't fail for a while. - */ - MPID_assert(rreq->mpid.uebuf == NULL); - MPID_assert(rreq->mpid.uebuflen == 0); - /* rreq->mpid.uebuf = NULL; */ - /* rreq->mpid.uebuflen = 0; */ -#ifdef OUT_OF_ORDER_HANDLING - if (MPIDI_In_cntr[source].n_OutOfOrderMsgs > 0) { - MPIDI_Recvq_process_out_of_order_msgs(source, context); - } -#endif - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } - /* ---------------------------------------- */ - /* Signal that the recv has been started. */ - /* ---------------------------------------- */ - MPIDI_Progress_signal(); -} - -/** - * \brief The callback for a new RZV RTS - * \param[in] context The context on which the message is being received. - * \param[in] cookie Unused - * \param[in] _msginfo The extended header information - * \param[in] msginfo_size The size of the extended header information - * \param[in] sndbuf Unused - * \param[in] sndlen Unused - * \param[in] sender The origin endpoint - * \param[out] recv Unused - */ -void -MPIDI_RecvRzvCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv == NULL); - MPID_assert(sndlen == 0); - MPIDI_RecvRzvCB_impl (context, sender, _msginfo, msginfo_size, 0); -} - -/** - * \brief The callback for a new "zero byte" RZV RTS - * \param[in] context The context on which the message is being received. - * \param[in] cookie Unused - * \param[in] _msginfo The extended header information - * \param[in] msginfo_size The size of the extended header information - * \param[in] sndbuf Unused - * \param[in] sndlen Unused - * \param[in] sender The origin endpoint - * \param[out] recv Unused - */ -void -MPIDI_RecvRzvCB_zerobyte(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv == NULL); - MPID_assert(sndlen == 0); - MPIDI_RecvRzvCB_impl (context, sender, _msginfo, msginfo_size, 1); -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_callback_short.c b/src/mpid/pamid/src/pt2pt/mpidi_callback_short.c deleted file mode 100644 index 8ecbef2..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_callback_short.c +++ /dev/null @@ -1,268 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_callback_short.c - * \brief The callback for a new short message - */ -#include <mpidimpl.h> - - -/** - * \brief The standard callback for a new message - * - * \param[in] context The context on which the message is being received. - * \param[in] cookie Unused - * \param[in] _msginfo The header information - * \param[in] msginfo_size The size of the header information - * \param[in] sndbuf If the message is short, this is the data - * \param[in] sndlen The size of the incoming data - * \param[out] recv If the message is long, this tells the message layer how to handle the data. - */ -static inline void -MPIDI_RecvShortCB(pami_context_t context, - const void * _msginfo, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - unsigned isSync) - __attribute__((__always_inline__)); -static inline void -MPIDI_RecvShortCB(pami_context_t context, - const void * _msginfo, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - unsigned isSync) -{ - MPID_assert(_msginfo != NULL); - - const MPIDI_MsgInfo *msginfo = (const MPIDI_MsgInfo *)_msginfo; - MPIR_Request * rreq = NULL; - pami_task_t source; -#if TOKEN_FLOW_CONTROL - int rettoks=0; -#endif - - /* -------------------- */ - /* Match the request. */ - /* -------------------- */ - unsigned rank = msginfo->MPIrank; - unsigned tag = msginfo->MPItag; - unsigned context_id = msginfo->MPIctxt; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - source = PAMIX_Endpoint_query(sender); - MPIDI_Receive_tokens(msginfo,source); -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDP(rank, tag, context_id); -#else - rreq = MPIDI_Recvq_FDP(rank, source, tag, context_id, msginfo->MPIseqno); -#endif - - /* Match not found */ - if (unlikely(rreq == NULL)) - { -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivals); -#endif - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIR_Request *newreq = MPIDI_Request_create2(); - MPID_assert(newreq != NULL); - if (sndlen) - { - newreq->mpid.uebuflen = sndlen; - if (!TOKEN_FLOW_CONTROL_ON) - { - newreq->mpid.uebuf = MPL_malloc(sndlen); - newreq->mpid.uebuf_malloc = mpiuMalloc; - } - else - { - #if TOKEN_FLOW_CONTROL - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - newreq->mpid.uebuf = MPIDI_mm_alloc(sndlen); - newreq->mpid.uebuf_malloc = mpidiBufMM; - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - #else - MPID_assert_always(0); - #endif - } - MPID_assert(newreq->mpid.uebuf != NULL); - } - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDP(rank, tag, context_id); -#else - rreq = MPIDI_Recvq_FDP(rank, PAMIX_Endpoint_query(sender), tag, context_id, msginfo->MPIseqno); -#endif - - if (unlikely(rreq == NULL)) - { - MPIDI_Callback_process_unexp(newreq, context, msginfo, sndlen, sender, sndbuf, NULL, isSync); - /* request is always complete now */ - if (TOKEN_FLOW_CONTROL_ON && sndlen) - { - #if TOKEN_FLOW_CONTROL - MPIDI_Token_cntr[source].unmatched++; - #else - MPID_assert_always(0); - #endif - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIR_Request_free(newreq); - goto fn_exit_short; - } - else - { - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPID_Request_discard(newreq); - } - } - else - { -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivalsMatched); -#endif - if (TOKEN_FLOW_CONTROL_ON && sndlen) - { - #if TOKEN_FLOW_CONTROL - MPIDI_Update_rettoks(source); - MPIDI_Must_return_tokens(context,source); - #else - MPID_assert_always(0); - #endif - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } - - /* the receive queue processing has been completed and we found match*/ - - /* ---------------------- */ - /* Copy in information. */ - /* ---------------------- */ - rreq->status.MPI_SOURCE = rank; - rreq->status.MPI_TAG = tag; - MPIR_STATUS_SET_COUNT(rreq->status, sndlen); - MPIDI_Request_setCA (rreq, MPIDI_CA_COMPLETE); - MPIDI_Request_cpyPeerRequestH(rreq, msginfo); - MPIDI_Request_setSync (rreq, isSync); - MPIDI_Request_setRzv (rreq, 0); - - /* ----------------------------- */ - /* Request was already posted. */ - /* ----------------------------- */ - if (unlikely(isSync)) - MPIDI_SyncAck_post(context, rreq, PAMIX_Endpoint_query(sender)); - - if (unlikely(HANDLE_GET_KIND(rreq->mpid.datatype) != HANDLE_KIND_BUILTIN)) - { - MPIDI_Callback_process_userdefined_dt(context, sndbuf, sndlen, rreq); - goto fn_exit_short; - } - - size_t dt_size = rreq->mpid.userbufcount * MPIDU_Datatype_get_basic_size(rreq->mpid.datatype); - - /* ----------------------------- */ - /* Test for truncated message. */ - /* ----------------------------- */ - if (unlikely(sndlen > dt_size)) - { -#if ASSERT_LEVEL > 0 - MPIDI_Callback_process_trunc(context, rreq, NULL, sndbuf); - goto fn_exit_short; -#else - sndlen = dt_size; -#endif - } - - MPID_assert(rreq->mpid.uebuf == NULL); - MPID_assert(rreq->mpid.uebuflen == 0); - void* rcvbuf = rreq->mpid.userbuf; - - if (sndlen > 0) - { -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on && MPIDI_cuda_is_device_buf(rcvbuf)) - { - cudaError_t cudaerr = CudaMemcpy(rcvbuf, sndbuf, (size_t)sndlen, cudaMemcpyHostToDevice); - } - else -#endif - memcpy(rcvbuf, sndbuf, sndlen); - } - TRACE_SET_R_VAL(source,(rreq->mpid.idx),rlen,sndlen); - TRACE_SET_R_BIT(source,(rreq->mpid.idx),fl.f.comp_in_HH); - TRACE_SET_R_VAL(source,(rreq->mpid.idx),bufadd,rreq->mpid.userbuf); - MPIDI_Request_complete(rreq); - - fn_exit_short: -#ifdef OUT_OF_ORDER_HANDLING - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - if (MPIDI_In_cntr[source].n_OutOfOrderMsgs>0) { - MPIDI_Recvq_process_out_of_order_msgs(source, context); - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); -#endif - - /* ---------------------------------------- */ - /* Signal that the recv has been started. */ - /* ---------------------------------------- */ - MPIDI_Progress_signal(); -} - - -void -MPIDI_RecvShortAsyncCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv == NULL); - MPID_assert(msginfo_size == sizeof(MPIDI_MsgInfo)); - MPIDI_RecvShortCB(context, - _msginfo, - sndbuf, - sndlen, - sender, - 0); -} - - -void -MPIDI_RecvShortSyncCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t msginfo_size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv == NULL); - MPID_assert(msginfo_size == sizeof(MPIDI_MsgInfo)); - MPIDI_RecvShortCB(context, - _msginfo, - sndbuf, - sndlen, - sender, - 1); -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_callback_util.c b/src/mpid/pamid/src/pt2pt/mpidi_callback_util.c deleted file mode 100644 index ca34ec5..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_callback_util.c +++ /dev/null @@ -1,195 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_callback_util.c - * \brief Utility functions to help uncommon cases in the new-message callbacks - */ -#include <mpidimpl.h> - - -/* MSGQUEUE lock must be held by caller */ -void -MPIDI_Callback_process_unexp(MPIR_Request *newreq, - pami_context_t context, - const MPIDI_MsgInfo * msginfo, - size_t sndlen, - pami_endpoint_t sender, - const void * sndbuf, - pami_recv_t * recv, - unsigned isSync) -{ - MPIR_Request *rreq = NULL; - - /* ---------------------------------------------------- */ - /* Fallback position: */ - /* + Request was not posted, or */ - /* + Request was long & not contiguous. */ - /* We must allocate enough space to hold the message. */ - /* The temporary buffer will be unpacked later. */ - /* ---------------------------------------------------- */ - unsigned rank = msginfo->MPIrank; - unsigned tag = msginfo->MPItag; - unsigned context_id = msginfo->MPIctxt; -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_AEU(newreq, rank, tag, context_id); -#else - unsigned msg_seqno = msginfo->MPIseqno; - rreq = MPIDI_Recvq_AEU(newreq, rank, PAMIX_Endpoint_query(sender), tag, context_id, msg_seqno); -#endif - /* ---------------------- */ - /* Copy in information. */ - /* ---------------------- */ - rreq->status.MPI_SOURCE = rank; - rreq->status.MPI_TAG = tag; - MPIR_STATUS_SET_COUNT(rreq->status, sndlen); - MPIDI_Request_setCA (rreq, MPIDI_CA_COMPLETE); - MPIDI_Request_cpyPeerRequestH(rreq, msginfo); - MPIDI_Request_setSync (rreq, isSync); - - /* Set the rank of the sender if a sync msg. */ -#ifndef OUT_OF_ORDER_HANDLING - if (isSync) - { -#endif - MPIDI_Request_setPeerRank_comm(rreq, rank); - MPIDI_Request_setPeerRank_pami(rreq, PAMIX_Endpoint_query(sender)); -#ifndef OUT_OF_ORDER_HANDLING - } -#endif - - MPID_assert(!sndlen || rreq->mpid.uebuf != NULL); - TRACE_MEMSET_R(PAMIX_Endpoint_query(sender),msg_seqno,recv_status); - TRACE_SET_R_VAL(PAMIX_Endpoint_query(sender),(msginfo->MPIseqno & SEQMASK),msgid,msginfo->MPIseqno); - TRACE_SET_R_VAL(PAMIX_Endpoint_query(sender),(msginfo->MPIseqno & SEQMASK),rtag,tag); - TRACE_SET_R_VAL(PAMIX_Endpoint_query(sender),(msginfo->MPIseqno & SEQMASK),rctx,msginfo->MPIctxt); - TRACE_SET_R_VAL(PAMIX_Endpoint_query(sender),(msginfo->MPIseqno & SEQMASK),rlen,sndlen); - TRACE_SET_R_VAL(PAMIX_Endpoint_query(sender),(msginfo->MPIseqno & SEQMASK),fl.f.sync,isSync); - TRACE_SET_R_VAL(PAMIX_Endpoint_query(sender),(msginfo->MPIseqno & SEQMASK),rsource,PAMIX_Endpoint_query(sender)); - TRACE_SET_REQ_VAL(rreq->mpid.idx,(msginfo->MPIseqno & SEQMASK)); - - if (recv != NULL) - { - recv->local_fn = MPIDI_RecvDoneCB_mutexed; - recv->cookie = rreq; - /* -------------------------------------------------- */ - /* Let PAMI know where to put the rest of the data. */ - /* -------------------------------------------------- */ - recv->addr = rreq->mpid.uebuf; - } - else - { - /* ------------------------------------------------- */ - /* We have the data; copy it and complete the msg. */ - /* ------------------------------------------------- */ - memcpy(rreq->mpid.uebuf, sndbuf, sndlen); - MPIDI_RecvDoneCB(context, rreq, PAMI_SUCCESS); - /* caller must release rreq, after unlocking MSGQUEUE */ - } -} - - -/* MSGQUEUE lock is not held */ -void -MPIDI_Callback_process_trunc(pami_context_t context, - MPIR_Request *rreq, - pami_recv_t *recv, - const void *sndbuf) -{ - rreq->status.MPI_ERROR = MPI_ERR_TRUNCATE; - - /* -------------------------------------------------------------- */ - /* The data is already available, so we can just unpack it now. */ - /* -------------------------------------------------------------- */ - if (recv) - { - MPIDI_Request_setCA(rreq, MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE); - rreq->mpid.uebuflen = MPIR_STATUS_GET_COUNT(rreq->status); - rreq->mpid.uebuf = MPL_malloc(MPIR_STATUS_GET_COUNT(rreq->status)); - MPID_assert(rreq->mpid.uebuf != NULL); - rreq->mpid.uebuf_malloc = mpiuMalloc; - - recv->addr = rreq->mpid.uebuf; - } - else - { - MPIDI_Request_setCA(rreq, MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE); - rreq->mpid.uebuflen = MPIR_STATUS_GET_COUNT(rreq->status); - rreq->mpid.uebuf = (void*)sndbuf; - MPIDI_RecvDoneCB(context, rreq, PAMI_SUCCESS); - MPIR_Request_free(rreq); - } -} - - -/* MSGQUEUE lock is not held */ -void -MPIDI_Callback_process_userdefined_dt(pami_context_t context, - const void * sndbuf, - size_t sndlen, - MPIR_Request * rreq) -{ - unsigned dt_contig, dt_size; - MPIDU_Datatype*dt_ptr; - MPI_Aint dt_true_lb; - MPIDI_Datatype_get_info(rreq->mpid.userbufcount, - rreq->mpid.datatype, - dt_contig, - dt_size, - dt_ptr, - dt_true_lb); - - /* ----------------------------- */ - /* Test for truncated message. */ - /* ----------------------------- */ - if (unlikely(sndlen > dt_size)) - { -#if ASSERT_LEVEL > 0 - MPIDI_Callback_process_trunc(context, rreq, NULL, sndbuf); - return; -#else - sndlen = dt_size; -#endif - } - - /* - * This is to test that the fields don't need to be - * initialized. Remove after this doesn't fail for a while. - */ - if (likely (dt_contig)) - { - MPID_assert(rreq->mpid.uebuf == NULL); - MPID_assert(rreq->mpid.uebuflen == 0); - void* rcvbuf = rreq->mpid.userbuf + dt_true_lb;; -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on && MPIDI_cuda_is_device_buf(rcvbuf)) - { - cudaError_t cudaerr = CudaMemcpy(rcvbuf, sndbuf, (size_t)sndlen, cudaMemcpyHostToDevice); - } - else -#endif - memcpy(rcvbuf, sndbuf, sndlen); - MPIDI_Request_complete(rreq); - return; - } - - MPIDI_Request_setCA(rreq, MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE); - rreq->mpid.uebuflen = sndlen; - rreq->mpid.uebuf = (void*)sndbuf; - MPIDI_RecvDoneCB(context, rreq, PAMI_SUCCESS); - MPIR_Request_free(rreq); -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_control.c b/src/mpid/pamid/src/pt2pt/mpidi_control.c deleted file mode 100644 index d2e2b87..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_control.c +++ /dev/null @@ -1,414 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_control.c - * \brief Interface to the control protocols used by MPID pt2pt - */ -#include <mpidimpl.h> - - -/** - * \brief Send a high-priority msginfo struct (control data) - * - * \param[in] control The pointer to the msginfo structure - * \param[in] peerrank The node to whom the control message is to be sent - */ -static inline void -MPIDI_CtrlSend(pami_context_t context, - MPIDI_MsgInfo * msginfo, - pami_task_t peerrank) -{ - pami_endpoint_t dest; - PAMI_Endpoint_create(MPIDI_Client, peerrank, 0, &dest); - - TRACE_ERR("CtrlSend: type=%d local=%u remote=%u\n", msginfo->control, MPIR_Process.comm_world->rank, peerrank); - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_Control, - .dest = dest, - .header = { - .iov_base = msginfo, - .iov_len = sizeof(MPIDI_MsgInfo), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - }; - - pami_result_t rc; - rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); -} - - -/** - * \brief Message layer callback which is invoked on the target node - * of a flow-control rendezvous operation. - * - * This callback is invoked when the data buffer on the origin node - * has been completely transfered to the target node. The target node - * must acknowledge the completion of the transfer to the origin node - * with a control message and then complete the receive by releasing - * the request object. - * - * \param[in,out] rreq MPI receive request object - */ -void -MPIDI_RecvRzvDoneCB(pami_context_t context, - void * cookie, - pami_result_t result) -{ - MPIR_Request * rreq = (MPIR_Request*)cookie; - MPID_assert(rreq != NULL); - - TRACE_ERR("RZV Done for req=%p addr=%p *addr[0]=%#016llx *addr[1]=%#016llx\n", - rreq, - rreq->mpid.userbuf, - *(((unsigned long long*)rreq->mpid.userbuf)+0), - *(((unsigned long long*)rreq->mpid.userbuf)+1)); - - /* Is it neccesary to save the original value of the 'type' field ?? */ - unsigned original_value = MPIDI_Request_getControl(rreq); - MPIDI_Request_setControl(rreq, MPIDI_CONTROL_RENDEZVOUS_ACKNOWLEDGE); - MPIDI_CtrlSend(context, - &rreq->mpid.envelope.msginfo, - MPIDI_Request_getPeerRank_pami(rreq)); - MPIDI_Request_setControl(rreq, original_value); - -#ifdef USE_PAMI_RDMA - pami_result_t rc; - rc = PAMI_Memregion_destroy(context, &rreq->mpid.memregion); - MPID_assert(rc == PAMI_SUCCESS); -#else - if( (!MPIDI_Process.mp_s_use_pami_get) && (rreq->mpid.memregion_used) ) - { - pami_result_t rc; - rc = PAMI_Memregion_destroy(context, &rreq->mpid.memregion); - MPID_assert(rc == PAMI_SUCCESS); - } -#endif - - MPIDI_RecvDoneCB(context, rreq, PAMI_SUCCESS); - MPIR_Request_free(rreq); -} - -/** - * \brief Message layer callback which is invoked on the target node - * of a 'zero-byte' flow-control rendezvous operation. - * - * \param[in] context Communication context - * \param[in] cookie Completion callback cookie - MPI receive request object - * \param[in] result Status - */ -void -MPIDI_RecvRzvDoneCB_zerobyte(pami_context_t context, - void * cookie, - pami_result_t result) -{ - MPIR_Request * rreq = (MPIR_Request*)cookie; - MPID_assert(rreq != NULL); - - /* Is it neccesary to save the original value of the 'type' field ?? */ - unsigned original_value = MPIDI_Request_getControl(rreq); - MPIDI_Request_setControl(rreq, MPIDI_CONTROL_RENDEZVOUS_ACKNOWLEDGE); - MPIDI_CtrlSend(context, - &rreq->mpid.envelope.msginfo, - MPIDI_Request_getPeerRank_pami(rreq)); - MPIDI_Request_setControl(rreq, original_value); - - MPIDI_RecvDoneCB(context, rreq, PAMI_SUCCESS); - TRACE_SET_R_BIT(MPIDI_Request_getPeerRank_pami(rreq),(rreq->mpid.idx),fl.f.sync_com_in_HH); - TRACE_SET_R_BIT(MPIDI_Request_getPeerRank_pami(rreq),(rreq->mpid.idx),fl.f.matchedInHH); - TRACE_SET_R_VAL(MPIDI_Request_getPeerRank_pami(rreq),(rreq->mpid.idx),bufadd,rreq->mpid.userbuf); - MPIR_Request_free(rreq); -} - -/** - * \brief Acknowledge an MPI_Ssend() - * - * \param[in] req The request element to acknowledge - * - * \return The same as MPIDI_CtrlSend() - */ -void -MPIDI_SyncAck_post(pami_context_t context, - MPIR_Request * req, - unsigned peer) -{ - MPIDI_Request_setControl(req, MPIDI_CONTROL_SSEND_ACKNOWLEDGE); - MPIDI_MsgInfo * info = &req->mpid.envelope.msginfo; - MPIDI_CtrlSend(context, info, peer); -} - - -/** - * \brief Acknowledge an MPI_Ssend() - * - * This is the handoff side, executing in advance(). - * The send is performed, and then the request is - * completed. The send is "immediate" so the payload - * has been copied upon return from MPIDI_CtrlSend(), - * so it is safe to complete the request. - * - * \param[in] context The PAMI context - * \param[in] req The request element to acknowledge - * - * \returns The PAMI return code - */ -pami_result_t -MPIDI_SyncAck_handoff(pami_context_t context, - void * inputReq) -{ - MPIR_Request *req = inputReq; - MPIDI_Request_setControl(req, MPIDI_CONTROL_SSEND_ACKNOWLEDGE); - MPIDI_MsgInfo * info = &req->mpid.envelope.msginfo; - unsigned peer = MPIDI_Request_getPeerRank_pami(req); - MPIDI_CtrlSend(context, info, peer); - MPIDI_Request_complete(req); - return PAMI_SUCCESS; -} - - -/** - * \brief Process an incoming MPI_Ssend() acknowledgment - * - * \param[in] info The contents of the control message as a MPIDI_MsgInfo struct - * \param[in] peer The rank of the node sending the data - */ -static inline void -MPIDI_SyncAck_proc(pami_context_t context, - const MPIDI_MsgInfo * info, - unsigned peer) -{ - MPID_assert(info != NULL); - MPIR_Request *req = MPIDI_Msginfo_getPeerRequest(info); - MPID_assert(req != NULL); - MPIDI_Request_complete(req); -} - - -/** - * \brief - * - * \param[in] context - * \param[in] req - */ -static inline void -MPIDI_RzvAck_proc_req(pami_context_t context, - MPIR_Request * req) -{ -#ifdef USE_PAMI_RDMA - pami_result_t rc; - rc = PAMI_Memregion_destroy(context, &req->mpid.envelope.memregion); - MPID_assert(rc == PAMI_SUCCESS); -#else - if( (!MPIDI_Process.mp_s_use_pami_get) && (req->mpid.envelope.memregion_used) ) - { - pami_result_t rc; - rc = PAMI_Memregion_destroy(context, &req->mpid.envelope.memregion); - MPID_assert(rc == PAMI_SUCCESS); - } -#endif - TRACE_SET_S_BIT(req->mpid.partner_id,(req->mpid.idx),fl.f.recvAck); - MPIDI_SendDoneCB(context, req, PAMI_SUCCESS); -} - - -/** - * \brief Process an incoming rendezvous acknowledgment from the - * target (remote) node and complete the MPI_Send() on the origin - * (local) node. - * - * \param[in] context - * \param[in] info The contents of the control message as a MPIDI_MsgInfo struct - * \param[in] peer The rank of the node sending the data - */ -static inline void -MPIDI_RzvAck_proc(pami_context_t context, - const MPIDI_MsgInfo * info, - pami_task_t peer) -{ - MPID_assert(info != NULL); - MPIR_Request *req = MPIDI_Msginfo_getPeerRequest(info); - MPID_assert(req != NULL); - MPIDI_RzvAck_proc_req(context, req); -} - - -/** - * \brief Process an incoming MPI_Send() cancelation - * - * \param[in] info The contents of the control message as a MPIDI_MsgInfo struct - * \param[in] peer The rank of the node sending the data - */ -static inline void -MPIDI_CancelReq_proc(pami_context_t context, - const MPIDI_MsgInfo * info, - pami_task_t peer) -{ - MPIDI_CONTROL type; - MPIDI_MsgInfo ackinfo; - MPIR_Request * sreq; - - MPID_assert(info != NULL); - - sreq=MPIDI_Recvq_FDUR(MPIDI_Msginfo_getPeerRequestH(info), - info->MPIrank, - info->MPItag, - info->MPIctxt); - if(sreq) - { - MPIR_Request_free(sreq); - type = MPIDI_CONTROL_CANCEL_ACKNOWLEDGE; - } - else - { - type = MPIDI_CONTROL_CANCEL_NOT_ACKNOWLEDGE; - } - - TRACE_ERR("Cancel search: {rank=%d:tag=%d:ctxt=%d:req=%d} my_request=%p result=%s\n", - info->MPIrank, - info->MPItag, - info->MPIctxt, - MPIDI_Msginfo_getPeerRequestH(info), - sreq, - (type==MPIDI_CONTROL_CANCEL_ACKNOWLEDGE) ? "ACK" : "NAK"); - - ackinfo.control = type; - MPIDI_Msginfo_cpyPeerRequestH(&ackinfo, info); - MPIDI_CtrlSend(context, &ackinfo, peer); -} - - -/** - * \brief Process an incoming MPI_Send() cancelation result - * - * \param[in] info The contents of the control message as a MPIDI_MsgInfo struct - * \param[in] peer The rank of the node sending the data - */ -static inline void -MPIDI_CancelAck_proc(pami_context_t context, - const MPIDI_MsgInfo * info, - pami_task_t peer) -{ - MPID_assert(info != NULL); - MPIR_Request *req = MPIDI_Msginfo_getPeerRequest(info); - MPID_assert(req != NULL); - - TRACE_ERR("Cancel result: my_request=%p result=%s\n", - req, - (info->control==MPIDI_CONTROL_CANCEL_ACKNOWLEDGE) ? "ACK" : "NAK"); - - if(info->control == MPIDI_CONTROL_CANCEL_NOT_ACKNOWLEDGE) - { - req->mpid.cancel_pending = FALSE; - } - else - { - MPID_assert(info->control == MPIDI_CONTROL_CANCEL_ACKNOWLEDGE); - MPID_assert(req->mpid.cancel_pending == TRUE); - - MPIR_STATUS_SET_CANCEL_BIT(req->status, TRUE); - - /* - * Rendezvous-Sends wait until a rzv ack is received to complete - * the send. Since this request was canceled, no rzv ack will be - * sent from the target node; fake the response here. - */ - if (MPIDI_Request_isRzv(req)) - { - TRACE_ERR("RZV\n"); - MPIDI_RzvAck_proc_req(context, req); - } - /* - * A canceled Sync-Send hasn't been ACKed (and now will never be - * acked). We call complete now to simulate an ACKed message. - * This is the entirety of the sync-ack processing, so it hasn't - * been made into a new function. - */ - if (MPIDI_Request_isSync(req)) - { - TRACE_ERR("Sync\n"); - MPIDI_Request_complete(req); - } - - /* - * Finally, this request has been faux-Sync-ACKed and - * faux-RZV-ACKed. We just do a normal completion. The user - * can call MPI_Wait()/MPI_Test() to finish it off (unless the - * message hasn't finished sending, in which case the done - * callback will finish it off). - */ - } - - TRACE_ERR("Completing request\n"); - MPIDI_Request_complete(req); -} - - -/** - * \brief This is the general PT2PT control message call-back - */ -void -MPIDI_ControlCB(pami_context_t context, - void * cookie, - const void * _msginfo, - size_t size, - const void * sndbuf, - size_t sndlen, - pami_endpoint_t sender, - pami_recv_t * recv) -{ - MPID_assert(recv == NULL); - MPID_assert(sndlen == 0); - MPID_assert(_msginfo != NULL); - MPID_assert(size == sizeof(MPIDI_MsgInfo)); - const MPIDI_MsgInfo *msginfo = (const MPIDI_MsgInfo *)_msginfo; - pami_task_t senderrank = PAMIX_Endpoint_query(sender); - - TRACE_ERR("CtrlRecv: type=%d local=%u remote=%u\n", msginfo->control, MPIR_Process.comm_world->rank, senderrank); - switch (msginfo->control) - { - case MPIDI_CONTROL_SSEND_ACKNOWLEDGE: - MPIDI_SyncAck_proc(context, msginfo, senderrank); - break; - case MPIDI_CONTROL_CANCEL_REQUEST: - MPIDI_CancelReq_proc(context, msginfo, senderrank); - break; - case MPIDI_CONTROL_CANCEL_ACKNOWLEDGE: - case MPIDI_CONTROL_CANCEL_NOT_ACKNOWLEDGE: - MPIDI_CancelAck_proc(context, msginfo, senderrank); - break; - case MPIDI_CONTROL_RENDEZVOUS_ACKNOWLEDGE: - MPIDI_RzvAck_proc(context, msginfo, senderrank); - break; -#if TOKEN_FLOW_CONTROL - case MPIDI_CONTROL_RETURN_TOKENS: - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - MPIDI_Token_cntr[sender].tokens += msginfo->alltokens; - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - break; -#endif - default: - fprintf(stderr, "Bad msginfo type: 0x%08x %d\n", - msginfo->control, - msginfo->control); - MPID_abort(); - } - MPIDI_Progress_signal(); -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_done.c b/src/mpid/pamid/src/pt2pt/mpidi_done.c deleted file mode 100644 index 8034b43..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_done.c +++ /dev/null @@ -1,316 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_done.c - * \brief "Done" call-backs provided to the message layer for signaling completion - */ -#include <mpidimpl.h> - - -/** - * \brief Message layer callback which is invoked on the origin node - * when the send of the message is done - * - * \param[in,out] sreq MPI receive request object - */ -void -MPIDI_SendDoneCB(pami_context_t context, - void * clientdata, - pami_result_t result) -{ - TRACE_SET_S_BIT((((MPIR_Request *) clientdata)->mpid.partner_id),(((MPIR_Request *) clientdata)->mpid.idx),fl.f.sendComp); - MPIDI_SendDoneCB_inline(context, - clientdata, - result); -} - - -static inline void -MPIDI_RecvDoneCB_copy(MPIR_Request * rreq) -{ - int smpi_errno; - MPID_assert(rreq->mpid.uebuf != NULL); - intptr_t _count=0; - MPIDI_Buffer_copy(rreq->mpid.uebuf, /* source buffer */ - rreq->mpid.uebuflen, - MPI_CHAR, - &smpi_errno, - rreq->mpid.userbuf, /* dest buffer */ - rreq->mpid.userbufcount, /* dest count */ - rreq->mpid.datatype, /* dest type */ - &_count, - &rreq->status.MPI_ERROR); - MPIR_STATUS_SET_COUNT(rreq->status, _count); -} - - -/** - * \brief Message layer callback which is invoked on the target node - * when the incoming message is complete. - * - * The MSGQUEUE lock may or may not be held. - * - * \param[in,out] rreq MPI receive request object - */ -void -MPIDI_RecvDoneCB(pami_context_t context, - void * clientdata, - pami_result_t result) -{ - MPIR_Request * rreq = (MPIR_Request*)clientdata; - MPID_assert(rreq != NULL); - switch (MPIDI_Request_getCA(rreq)) - { - case MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE: - { - MPIDI_RecvDoneCB_copy(rreq); - /* free the unexpected data buffer later */ - break; - } - case MPIDI_CA_COMPLETE: - { - break; - } - default: - { - MPID_Abort(NULL, MPI_ERR_OTHER, -1, "Internal: unknown CA"); - break; - } - } -#ifdef OUT_OF_ORDER_HANDLING - MPIR_Request * oo_peer = rreq->mpid.oo_peer; - if (oo_peer) { - MPIR_STATUS_SET_COUNT(oo_peer->status, MPIR_STATUS_GET_COUNT(rreq->status)); - MPIDI_Request_complete(oo_peer); - } -#endif - MPIDI_Request_complete_norelease(rreq); - /* caller must release rreq, after unlocking MSGQUEUE (if held) */ -#ifdef OUT_OF_ORDER_HANDLING - pami_task_t source; - source = MPIDI_Request_getPeerRank_pami(rreq); - if (MPIDI_In_cntr[source].n_OutOfOrderMsgs > 0) { - MPIDI_Recvq_process_out_of_order_msgs(source, context); - } -#endif -} - - -/** - * \brief Thread-safe message layer callback which is invoked on the - * target node when the incoming message is complete. - * - * \param[in,out] rreq MPI receive request object - */ -void -MPIDI_RecvDoneCB_mutexed(pami_context_t context, - void * clientdata, - pami_result_t result) -{ - MPIR_Request * rreq = (MPIR_Request*)clientdata; - MPIU_THREAD_CS_ENTER(MSGQUEUE, 0); - - MPIDI_RecvDoneCB(context, clientdata, result); - - MPIU_THREAD_CS_EXIT(MSGQUEUE, 0); - MPIR_Request_free(rreq); -} - - -#ifdef OUT_OF_ORDER_HANDLING -/** - * \brief Checks if any of the messages in the out of order list is ready - * to be processed, if so, process it - */ -void MPIDI_Recvq_process_out_of_order_msgs(pami_task_t src, pami_context_t context) -{ - /*******************************************************/ - /* If the next message to be processed in the */ - /* a recv is posted, then process the message. */ - /*******************************************************/ - MPIDI_In_cntr_t *in_cntr; - MPIR_Request *ooreq, *rreq, *prev_rreq; - pami_get_simple_t xferP; - intptr_t _count=0; - int matched; - void * it; - - in_cntr = &MPIDI_In_cntr[src]; - prev_rreq = NULL; - ooreq = in_cntr->OutOfOrderList; - while((in_cntr->n_OutOfOrderMsgs !=0) && ((MPIDI_Request_getMatchSeq(ooreq) == (in_cntr->nMsgs+1)) || (MPIDI_Request_getMatchSeq(ooreq) == in_cntr->nMsgs))) - { - matched=0; - matched=MPIDI_Search_recv_posting_queue(MPIDI_Request_getMatchRank(ooreq),MPIDI_Request_getMatchTag(ooreq),MPIDI_Request_getMatchCtxt(ooreq),&rreq, &it); - - if (matched) { - /* process a completed message i.e. data is in EA */ - if (TOKEN_FLOW_CONTROL_ON) { - #if TOKEN_FLOW_CONTROL - if ((ooreq->mpid.uebuflen) && (!(ooreq->mpid.envelope.msginfo.isRzv))) { - MPIDI_Token_cntr[src].unmatched--; - MPIDI_Update_rettoks(src); - } - MPIDI_Must_return_tokens(context,src); - #else - MPID_assert_always(0); - #endif - } - if (MPIDI_Request_getMatchSeq(ooreq) == (in_cntr->nMsgs+ 1)) - in_cntr->nMsgs++; - - if (ooreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - MPIDI_Recvq_remove_req_from_ool(ooreq,in_cntr); - } - - /* ----------------------------------------- */ - /* Calculate message length for reception. */ - /* ----------------------------------------- */ - unsigned dt_contig, dt_size; - MPIDU_Datatype*dt_ptr; - MPI_Aint dt_true_lb; - MPIDI_Datatype_get_info(rreq->mpid.userbufcount, - rreq->mpid.datatype, - dt_contig, - dt_size, - dt_ptr, - dt_true_lb); - if (unlikely(ooreq->mpid.uebuflen > dt_size)) - { - MPIR_STATUS_SET_COUNT(rreq->status, dt_size); - rreq->status.MPI_ERROR = MPI_ERR_TRUNCATE; - } - - TRACE_SET_REQ_VAL(rreq->mpid.idx,ooreq->mpid.idx); - TRACE_SET_R_BIT(src,(rreq->mpid.idx),fl.f.matchedInOOL); - TRACE_SET_R_VAL(src,(rreq->mpid.idx),rlen,dt_size); - ooreq->comm = rreq->comm; - MPIR_Comm_add_ref(ooreq->comm); - ooreq->mpid.userbuf = rreq->mpid.userbuf; - ooreq->mpid.userbufcount = rreq->mpid.userbufcount; - ooreq->mpid.datatype = rreq->mpid.datatype; - if (HANDLE_GET_KIND(ooreq->mpid.datatype) != HANDLE_KIND_BUILTIN) - { - MPIDU_Datatype_get_ptr(ooreq->mpid.datatype, ooreq->mpid.datatype_ptr); - MPIDU_Datatype_add_ref(ooreq->mpid.datatype_ptr); - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - MPIDI_Recvq_remove_uexp_noit(MPIDI_Request_getMatchRank(ooreq),MPIDI_Request_getMatchTag(ooreq),MPIDI_Request_getMatchCtxt(ooreq), MPIDI_Request_getMatchSeq(ooreq)); - else -#endif - MPIDI_Recvq_remove(MPIDI_Recvq.unexpected, ooreq, ooreq->mpid.prev); - if (!MPIR_cc_is_complete(&ooreq->cc)) { - ooreq->mpid.oo_peer = rreq; - MPIDI_RecvMsg_Unexp(ooreq, rreq->mpid.userbuf, rreq->mpid.userbufcount, rreq->mpid.datatype); - } else { - MPIDI_RecvMsg_Unexp(ooreq, rreq->mpid.userbuf, rreq->mpid.userbufcount, rreq->mpid.datatype); - MPIR_STATUS_SET_COUNT(rreq->status, MPIR_STATUS_GET_COUNT(ooreq->status)); - rreq->status.MPI_SOURCE = ooreq->status.MPI_SOURCE; - rreq->status.MPI_TAG = ooreq->status.MPI_TAG; - rreq->mpid.envelope.msginfo.MPIseqno = ooreq->mpid.envelope.msginfo.MPIseqno; - MPIDI_Request_complete(rreq); - } - MPIR_Request_free(ooreq); - - } else { - if (MPIDI_Request_getMatchSeq(ooreq) == (in_cntr->nMsgs+ 1)) - in_cntr->nMsgs++; - if (ooreq->mpid.nextR != NULL) { /* recv is in the out of order list */ - MPIDI_Recvq_remove_req_from_ool(ooreq,in_cntr); - } - } - if (in_cntr->n_OutOfOrderMsgs > 0) - ooreq = in_cntr->OutOfOrderList; - } /* while */ -} - - -/** - * \brief search the posted recv queue and if found eliminate the - * element from the queue and return in *request; else return NULL - */ -int MPIDI_Search_recv_posting_queue(int src, int tag, int context_id, - MPIR_Request **request, void** it ) -{ - MPIR_Request * rreq; - MPIR_Request * prev_rreq = NULL; - - *request = NULL; -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - if(MPIDI_Process.queue_binary_search_support_on) - { - MPIDI_Recvq_find_in_post(src, tag, context_id, &rreq, it); - if (rreq != NULL) - { - /* The communicator test is not yet correct */ - MPIDI_Recvq_remove_post(src, tag, context_id, *it); - *request = rreq; -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivalsMatched); -#endif - return 1; - } - else - { - MPIDI_Recvq_find_in_post(src, MPI_ANY_TAG, context_id, &rreq, it); - if (rreq == NULL) - { - MPIDI_Recvq_find_in_post(MPI_ANY_SOURCE, tag, context_id, &rreq, it); - if (rreq == NULL) - MPIDI_Recvq_find_in_post(MPI_ANY_SOURCE, MPI_ANY_TAG, context_id, &rreq, it); - } - if (rreq != NULL) - { - /* The communicator test is not yet correct */ - MPIDI_Recvq_remove_post(MPIDI_Request_getMatchRank(rreq), MPIDI_Request_getMatchTag(rreq), context_id, *it); - *request = rreq; -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivalsMatched); -#endif - return 1; - } - } - } - else - { -#endif - rreq = MPIDI_Recvq.posted_head; - while (rreq != NULL) - { - /* The communicator test is not yet correct */ - if ((MPIDI_Request_getMatchRank(rreq)==src || MPIDI_Request_getMatchRank(rreq)==MPI_ANY_SOURCE) && - (MPIDI_Request_getMatchCtxt(rreq)==context_id) && - (MPIDI_Request_getMatchTag(rreq) == tag || MPIDI_Request_getMatchTag(rreq) == MPI_ANY_TAG) - ) { - MPIDI_Recvq_remove(MPIDI_Recvq.posted, rreq, prev_rreq); - *request = rreq; -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->earlyArrivalsMatched); -#endif - return 1; - } - prev_rreq = rreq; - rreq = rreq->mpid.next; - } -#ifdef QUEUE_BINARY_SEARCH_SUPPORT - } -#endif - return 0; -} -#endif diff --git a/src/mpid/pamid/src/pt2pt/mpidi_recv.h b/src/mpid/pamid/src/pt2pt/mpidi_recv.h deleted file mode 100644 index f2c1594..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_recv.h +++ /dev/null @@ -1,257 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_recv.h - * \brief ADI level implemenation of MPI_Irecv() - */ - -#ifndef __src_pt2pt_mpidi_recv_h__ -#define __src_pt2pt_mpidi_recv_h__ - -#include <mpidimpl.h> -#include "../mpid_recvq.h" -#include "mpidu_datatype.h" -/*#ifdef MPIDI_STATISTICS - #include "../../include/mpidi_datatypes.h" -#endif*/ - - -#if TOKEN_FLOW_CONTROL -extern MPIDI_Out_cntr_t *MPIDI_Out_cntr; -extern int MPIDI_tfctrl_hwmark; -extern void *MPIDI_mm_alloc(size_t); -extern void MPIDI_mm_free(void *, size_t); -extern int tfctrl_enabled; -extern char *EagerLimit; -#define MPIDI_Return_tokens MPIDI_Return_tokens_inline -#define MPIDI_Receive_tokens MPIDI_Receive_tokens_inline -#define MPIDI_Update_rettoks MPIDI_Update_rettoks_inline -#define MPIDI_Alloc_lock MPIDI_Alloc_lock_inline -#define MPIDI_Must_return_tokens MPIDI_Must_return_tokens_inline - -static inline void * -MPIDI_Return_tokens_inline(pami_context_t context, int dest, int tokens) -{ - MPIDI_MsgInfo tokenInfo; - if (tokens) { - memset(&tokenInfo,0, sizeof(MPIDI_MsgInfo)); - tokenInfo.control=MPIDI_CONTROL_RETURN_TOKENS; - tokenInfo.alltokens=tokens; - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_Control, - .dest = dest, - .header = { - .iov_base = &tokenInfo, - .iov_len = sizeof(MPIDI_MsgInfo), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - }; - pami_result_t rc; - rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - } -} - - -static inline void * -MPIDI_Must_return_tokens_inline(pami_context_t context,int dest) -{ - int rettoks=0; - - if (MPIDI_Token_cntr[dest].rettoks - && (MPIDI_Token_cntr[dest].rettoks + MPIDI_Token_cntr[dest].unmatched - >= MPIDI_tfctrl_hwmark)) - { - rettoks=MPIDI_Token_cntr[dest].rettoks; - MPIDI_Token_cntr[dest].rettoks=0; - MPIDI_Return_tokens_inline(context,dest,rettoks); - - } -} - -static inline void * -MPIDI_Receive_tokens_inline(const MPIDI_MsgInfo *m, int dest) - { - if ((m)->tokens) - { - MPIDI_Token_cntr[dest].tokens += (m)->tokens; - } - } - -static inline void * -MPIDI_Update_rettoks_inline(int source) - { - MPIDI_Token_cntr[source].rettoks++; - } - -static inline void * -MPIDI_Alloc_lock_inline(void **buf,size_t size) - { - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - (*buf) = (void *) MPIDI_mm_alloc(size); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } - -#else -#define MPIDI_Return_tokens(x,y,z) -#define MPIDI_Receive_tokens(x,y) -#define MPIDI_Update_rettoks(x) -#define MPIDI_Must_return_tokens(x,y) (0) -#define MPIDI_Alloc_lock(x,y) -#endif - -/** - * \brief ADI level implemenation of MPI_(I)Recv() - * - * \param[in] buf The buffer to receive into - * \param[in] count Number of expected elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The sending rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[out] status Update the status structure - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - */ -static inline int -MPIDI_Recv(void * buf, - MPI_Aint count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - unsigned is_blocking, - MPI_Status * status, - MPIR_Request ** request) -{ - MPIR_Request * rreq; - int found; - int mpi_errno = MPI_SUCCESS; - - /* ---------------------------------------- */ - /* NULL rank means empty request */ - /* ---------------------------------------- */ - if (unlikely(rank == MPI_PROC_NULL)) - { - MPIDI_RecvMsg_procnull(comm, is_blocking, status, request); - return MPI_SUCCESS; - } -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->recvs); -#endif - MPIR_Comm_add_ref(comm); - - /* ---------------------------------------- */ - /* find our request in the unexpected queue */ - /* or allocate one in the posted queue */ - /* ---------------------------------------- */ - MPIR_Request *newreq = MPIDI_Request_create2(); - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); -#ifndef OUT_OF_ORDER_HANDLING - rreq = MPIDI_Recvq_FDU_or_AEP(newreq, rank, - tag, - comm->recvcontext_id + context_offset, - &found); -#else - int pami_source; - if(rank != MPI_ANY_SOURCE) { - pami_source = MPID_VCR_GET_LPID(comm->vcr, rank); - } else { - pami_source = MPI_ANY_SOURCE; - } - if ((pami_source != MPI_ANY_SOURCE) && (MPIDI_In_cntr[pami_source].n_OutOfOrderMsgs>0)) { - /* returns unlock */ - MPIDI_Recvq_process_out_of_order_msgs(pami_source, MPIDI_Context[0]); - } - rreq = MPIDI_Recvq_FDU_or_AEP(newreq, rank, - pami_source, - tag, - comm->recvcontext_id + context_offset, - &found); -#endif - MPIDI_SET_PR_REC(rreq,buf,count,datatype,pami_source,rank,tag,comm,is_blocking); - - /* ----------------------------------------------------------------- */ - /* populate request with our data */ - /* We can do this because this is not a multithreaded implementation */ - /* ----------------------------------------------------------------- */ - - rreq->comm = comm; - rreq->mpid.userbuf = buf; - rreq->mpid.userbufcount = count; - rreq->mpid.datatype = datatype; - /* We don't need this because MPIDI_CA_COMPLETE is the initialized default */ - /* MPIDI_Request_setCA(rreq, MPIDI_CA_COMPLETE); */ - - if (unlikely(found)) - { - TRACE_SET_R_VALX(pami_source,rreq,len,rreq->mpid.uebuflen); - MPIDI_RecvMsg_Unexp(rreq, buf, count, datatype); - mpi_errno = rreq->status.MPI_ERROR; - if (TOKEN_FLOW_CONTROL_ON) { - #if TOKEN_FLOW_CONTROL - if ((rreq->mpid.uebuflen) && (!(rreq->mpid.envelope.msginfo.isRzv))) { - MPIDI_Token_cntr[(rreq->mpid.peer_pami)].unmatched--; - MPIDI_Update_rettoks(rreq->mpid.peer_pami); - } - MPIDI_Must_return_tokens(MPIDI_Context[0],(rreq->mpid.peer_pami)); - #else - MPID_assert_always(0); - #endif - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPID_Request_discard(newreq); -#ifdef OUT_OF_ORDER_HANDLING - if ((MPIDI_In_cntr[rreq->mpid.peer_pami].n_OutOfOrderMsgs>0)) - MPIDI_Recvq_process_out_of_order_msgs(rreq->mpid.peer_pami, MPIDI_Context[0]); -#endif - } - else - { - /* ----------------------------------------------------------- */ - /* request not found in unexpected queue, allocated and posted */ - /* ----------------------------------------------------------- */ - if (HANDLE_GET_KIND(datatype) != HANDLE_KIND_BUILTIN) - { - MPIDU_Datatype_get_ptr(datatype, rreq->mpid.datatype_ptr); - MPIDU_Datatype_add_ref(rreq->mpid.datatype_ptr); - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - } - - /* mutex has been dropped... */ - *request = rreq; - if (status != MPI_STATUS_IGNORE) - *status = rreq->status; -#ifdef MPIDI_STATISTICS - if (!(MPIR_cc_is_complete(&rreq->cc))) - { - MPID_NSTAT(mpid_statp->recvWaitsComplete); - } -#endif - - return mpi_errno; -} - -#endif diff --git a/src/mpid/pamid/src/pt2pt/mpidi_recvmsg.c b/src/mpid/pamid/src/pt2pt/mpidi_recvmsg.c deleted file mode 100644 index 16092e6..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_recvmsg.c +++ /dev/null @@ -1,161 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_recvmsg.c - * \brief ADI level implemenation of common recv code. - */ -#include <mpidimpl.h> -#include <mpidi_macros.h> - - -void -MPIDI_RecvMsg_Unexp(MPIR_Request * rreq, - void * buf, - MPI_Aint count, - MPI_Datatype datatype) -{ - /* ------------------------------------------------------------ */ - /* message was found in unexpected queue */ - /* ------------------------------------------------------------ */ - /* We must acknowledge synchronous send requests */ - /* The recvnew callback will acknowledge the posted messages */ - /* Recv functions will ack the messages that are unexpected */ - /* ------------------------------------------------------------ */ - TRACE_SET_R_BIT((rreq->mpid.partner_id),(rreq->mpid.idx),fl.f.matchedInUQ); - - if (MPIDI_Request_isRzv(rreq)) - { - const unsigned is_sync = MPIDI_Request_isSync(rreq); - const unsigned is_zero = (rreq->mpid.envelope.length==0); - - /* -------------------------------------------------------- */ - /* Received an expected flow-control rendezvous RTS. */ - /* This is very similar to the found/incomplete case */ - /* -------------------------------------------------------- */ - if (HANDLE_GET_KIND(datatype) != HANDLE_KIND_BUILTIN) - { - MPIDU_Datatype_get_ptr(datatype, rreq->mpid.datatype_ptr); - MPIDU_Datatype_add_ref(rreq->mpid.datatype_ptr); - } - - if (likely((is_sync+is_zero) == 0)) - MPIDI_Context_post(MPIDI_Context_local(rreq), &rreq->mpid.post_request, MPIDI_RendezvousTransfer, rreq); - else if (is_sync != 0) - MPIDI_Context_post(MPIDI_Context_local(rreq), &rreq->mpid.post_request, MPIDI_RendezvousTransfer_SyncAck, rreq); - else - MPIDI_Context_post(MPIDI_Context_local(rreq), &rreq->mpid.post_request, MPIDI_RendezvousTransfer_zerobyte, rreq); - } - else - { - if (MPIR_cc_is_complete(&rreq->cc)) - { - if (unlikely(MPIDI_Request_isSync(rreq))) - { - /* Post this to the context for asynchronous progresss. We cannot do - * the send-immediate inline here because we may not have the - * context locked (its is being asynchrously advanced). - * Must "uncomplete" the message (increment the ref and completion counts) so we - * hold onto this request object until this send has completed. When MPIDI_SyncAck_handoff - * finishes sending the ack, it will complete the request, decrementing the ref and - * completion counts. - */ - MPIDI_Request_uncomplete(rreq); - MPIDI_Send_post(MPIDI_SyncAck_handoff, rreq); - } - /* -------------------------------- */ - /* request is complete */ - /* -------------------------------- */ - if (rreq->mpid.uebuf != NULL) - { - if (likely(MPIR_STATUS_GET_CANCEL_BIT(rreq->status) == FALSE)) - { - intptr_t _count=0; - MPIDI_Buffer_copy(rreq->mpid.uebuf, - rreq->mpid.uebuflen, - MPI_CHAR, - &rreq->status.MPI_ERROR, - buf, - count, - datatype, - &_count, - &rreq->status.MPI_ERROR); - MPIR_STATUS_SET_COUNT(rreq->status, _count); - } - } - else - { - MPID_assert(rreq->mpid.uebuflen == 0); - MPIR_STATUS_SET_COUNT(rreq->status, 0); - } - } - else - { - /* -------------------------------- */ - /* request is incomplete */ - /* -------------------------------- */ - if (unlikely(MPIDI_Request_isSync(rreq))) - { - /* Post this to the context for asynchronous progresss. We cannot do - * the send-immediate inline here because we may not have the - * context locked (its is being asynchrously advanced). - * Must "uncomplete" the message (increment the ref and completion counts) so we - * hold onto this request object until this send has completed. When MPIDI_SyncAck_handoff - * finishes sending the ack, it will complete the request, decrementing the ref and - * completion counts. - */ - MPIDI_Request_uncomplete(rreq); - MPIDI_Send_post(MPIDI_SyncAck_handoff, rreq); - } - if (MPIR_STATUS_GET_CANCEL_BIT(rreq->status) == FALSE) - { - MPIDI_Request_setCA(rreq, MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE); - } - if (HANDLE_GET_KIND(datatype) != HANDLE_KIND_BUILTIN) - { - MPIDU_Datatype_get_ptr(datatype, rreq->mpid.datatype_ptr); - MPIDU_Datatype_add_ref(rreq->mpid.datatype_ptr); - } - } - } -} - - - -void -MPIDI_RecvMsg_procnull(MPIR_Comm * comm, - unsigned is_blocking, - MPI_Status * status, - MPIR_Request ** request) -{ - if (is_blocking) - { - MPIR_Status_set_procnull(status); - *request = NULL; - } - else - { - MPIR_Request * rreq; - rreq = MPIDI_Request_create2(); - MPIR_Status_set_procnull(&rreq->status); - rreq->kind = MPIR_REQUEST_KIND__RECV; - rreq->comm = comm; - MPIR_Comm_add_ref(comm); - MPIDI_Request_complete(rreq); - *request = rreq; - } -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_rendezvous.c b/src/mpid/pamid/src/pt2pt/mpidi_rendezvous.c deleted file mode 100644 index a218865..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_rendezvous.c +++ /dev/null @@ -1,266 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_rendezvous.c - * \brief Provide for a flow-control rendezvous-based send - */ -#include <mpidimpl.h> - -inline void -MPIDI_RendezvousTransfer_use_pami_rget(pami_context_t context, - pami_endpoint_t dest, - MPIR_Request *rreq) -__attribute__((__always_inline__)); -#ifdef RDMA_FAILOVER -inline void -MPIDI_RendezvousTransfer_use_pami_get(pami_context_t context, - pami_endpoint_t dest, - void *rcvbuf, - MPIR_Request *rreq) -__attribute__((__always_inline__)); -#endif - - -pami_result_t -MPIDI_RendezvousTransfer(pami_context_t context, - void * _rreq) -{ - MPIR_Request * rreq = (MPIR_Request*) _rreq; - - void *rcvbuf; - size_t rcvlen; - - /* -------------------------------------- */ - /* calculate message length for reception */ - /* calculate receive message "count" */ - /* -------------------------------------- */ - unsigned dt_contig; - size_t dt_size; - MPIDU_Datatype*dt_ptr; - MPI_Aint dt_true_lb; - MPIDI_Datatype_get_info(rreq->mpid.userbufcount, - rreq->mpid.datatype, - dt_contig, - dt_size, - dt_ptr, - dt_true_lb); - - /* -------------------------------------- */ - /* test for truncated message. */ - /* -------------------------------------- */ - if (rreq->mpid.envelope.length > dt_size) - { - rcvlen = dt_size; - rreq->status.MPI_ERROR = MPI_ERR_TRUNCATE; - MPIR_STATUS_SET_COUNT(rreq->status, rcvlen); - } - else - { - rcvlen = rreq->mpid.envelope.length; - } - - /* -------------------------------------- */ - /* if buffer is contiguous ... */ - /* -------------------------------------- */ - if (dt_contig) - { - MPIDI_Request_setCA(rreq, MPIDI_CA_COMPLETE); - rreq->mpid.uebuf = NULL; - rreq->mpid.uebuflen = 0; - rcvbuf = rreq->mpid.userbuf + dt_true_lb; - } - - /* --------------------------------------------- */ - /* buffer is non-contiguous. we need to allocate */ - /* a temporary buffer, and unpack later. */ - /* --------------------------------------------- */ - else - { - MPIDI_Request_setCA(rreq, MPIDI_CA_UNPACK_UEBUF_AND_COMPLETE); - rcvbuf = MPL_malloc(rcvlen); - MPID_assert(rcvbuf != NULL); - rreq->mpid.uebuf = rcvbuf; - rreq->mpid.uebuflen = rcvlen; - rreq->mpid.uebuf_malloc = mpiuMalloc; - } - - /* ---------------------------------------------------------------- */ - /* Get the data from the origin node. */ - /* ---------------------------------------------------------------- */ - - pami_result_t rc; - pami_endpoint_t dest; - MPIDI_Context_endpoint(rreq, &dest); - -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on && MPIDI_cuda_is_device_buf(rcvbuf)) - { - MPIDI_RendezvousTransfer_use_pami_get(context,dest,rcvbuf,rreq); - } - else - { -#endif - - -#ifdef USE_PAMI_RDMA - size_t rcvlen_out; - rc = PAMI_Memregion_create(context, - rcvbuf, - rcvlen, - &rcvlen_out, - &rreq->mpid.memregion); - MPID_assert(rc == PAMI_SUCCESS); - MPID_assert(rcvlen == rcvlen_out); - - TRACE_ERR("RZV Xfer for req=%p addr=%p *addr[0]=%#016llx *addr[1]=%#016llx\n", - rreq, - rcvbuf, - *(((unsigned long long*)rcvbuf)+0), - *(((unsigned long long*)rcvbuf)+1)); - - MPIDI_RendezvousTransfer_use_pami_rget(context,dest,rreq); -#else - rreq->mpid.memregion_used=0; - if( (!MPIDI_Process.mp_s_use_pami_get) && (rreq->mpid.envelope.memregion_used) ) - { - size_t rcvlen_out; - rc = PAMI_Memregion_create(context, - rcvbuf, - rcvlen, - &rcvlen_out, - &rreq->mpid.memregion); - if (rc == PAMI_SUCCESS) - { - rreq->mpid.memregion_used=1; - MPID_assert(rcvlen == rcvlen_out); - - TRACE_ERR("RZV Xfer for req=%p addr=%p *addr[0]=%#016llx *addr[1]=%#016llx\n", - rreq, - rcvbuf, - *(((unsigned long long*)rcvbuf)+0), - *(((unsigned long long*)rcvbuf)+1)); - MPIDI_RendezvousTransfer_use_pami_rget(context,dest,rreq); - } else { - MPIDI_RendezvousTransfer_use_pami_get(context,dest,rcvbuf,rreq); - } - } else { - MPIDI_RendezvousTransfer_use_pami_get(context,dest,rcvbuf,rreq); - } -#endif - -#if CUDA_AWARE_SUPPORT - } -#endif - - return PAMI_SUCCESS; -} - - -inline void -MPIDI_RendezvousTransfer_use_pami_rget(pami_context_t context, - pami_endpoint_t dest, - MPIR_Request *rreq) -{ - pami_result_t rc; - pami_rget_simple_t params = { - .rma = { - .dest = dest, - .hints = { - .buffer_registered = PAMI_HINT_ENABLE, - .use_rdma = PAMI_HINT_ENABLE, - .remote_async_progress = PAMI_HINT_DEFAULT, - .use_shmem = PAMI_HINT_DEFAULT, - }, - .bytes = rreq->mpid.envelope.length, - .cookie = rreq, - .done_fn = MPIDI_RecvRzvDoneCB, - }, - .rdma = { - .local = { - .mr = &rreq->mpid.memregion, - .offset = 0, - }, - .remote = { - .mr = &rreq->mpid.envelope.memregion, - .offset = 0, - }, - }, - }; - - rc = PAMI_Rget(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); -} - - -#ifdef RDMA_FAILOVER -inline void -MPIDI_RendezvousTransfer_use_pami_get(pami_context_t context, - pami_endpoint_t dest, - void *rcvbuf, - MPIR_Request *rreq) -{ - pami_result_t rc; - int val=0; - - if (MPIDI_Process.mp_s_use_pami_get) - val=PAMI_HINT_DEFAULT; - else - val=PAMI_HINT_DISABLE; - pami_get_simple_t params = { - .rma = { - .dest = dest, - .hints = { - .buffer_registered = PAMI_HINT_DEFAULT, - .use_rdma = val, - .remote_async_progress = PAMI_HINT_DEFAULT, - .use_shmem = PAMI_HINT_DEFAULT, -#ifndef OUT_OF_ORDER_HANDLING - .no_long_header= 1, -#endif - }, - .bytes = rreq->mpid.envelope.length, - .cookie = rreq, - .done_fn = MPIDI_RecvRzvDoneCB, - }, - .addr = { - .local = rcvbuf, - .remote = rreq->mpid.envelope.data, - }, - }; - - rc = PAMI_Get(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); -} -#endif - -pami_result_t MPIDI_RendezvousTransfer_SyncAck (pami_context_t context, void * _rreq) -{ - MPIR_Request *rreq = (MPIR_Request*)_rreq; - - // Do the sync ack transfer here. - MPIDI_SyncAck_post (context, rreq, MPIDI_Request_getPeerRank_pami(rreq)); - - // Continue on to the rendezvous transfer part. - return MPIDI_RendezvousTransfer(context, _rreq); -} - -pami_result_t MPIDI_RendezvousTransfer_zerobyte (pami_context_t context, void * _rreq) -{ - MPIDI_RecvRzvDoneCB_zerobyte (context, _rreq, PAMI_SUCCESS); - return PAMI_SUCCESS; -} diff --git a/src/mpid/pamid/src/pt2pt/mpidi_send.h b/src/mpid/pamid/src/pt2pt/mpidi_send.h deleted file mode 100644 index b061771..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_send.h +++ /dev/null @@ -1,128 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_send.h - * - * This is a generic inline send operator. It groups code previous - * found in mpid_{,i}{,s}send.c. The hope is that the common code - * will be easier to maintain, and that the inline nature of the flags - * will ensure high performance. - */ -#include <mpidimpl.h> - -#ifndef __src_pt2pt_mpidi_send_h__ -#define __src_pt2pt_mpidi_send_h__ - -#include <mpidi_macros.h> - -static inline void -MPIDI_SendDoneCB_inline(pami_context_t context, - void * clientdata, - pami_result_t result) -{ - MPIR_Request * sreq = (MPIR_Request*)clientdata; - MPID_assert(sreq != NULL); - MPIDI_Request_complete(sreq); -} - - -/** - * \brief This is a generic inline verion of the various send functions. - * - * This is the same function prototype as MPID_{,I}{,S}send.c, but it - * inludes two inline flags: is_blocking and is_sync. These are not - * meant to be truely variable, but are expected to be known at - * compile time. That *should* allow the compiler to remove any extra - * code from this function that won't be used in the specific inlined - * version. - * - * The PAMI_Context_post function may queue this send operation for - * later handling and then return to the user. Therefore, this - * function has only two goals: - * - * + Fill in the request oject with all relevant information, and - * + Do any reference counting that must be done before the function returns. - * - * - * \param[in] buf The buffer to send - * \param[in] count Number of elements in the buffer - * \param[in] datatype The datatype of each element - * \param[in] rank The destination rank - * \param[in] tag The message tag - * \param[in] comm Pointer to the communicator - * \param[in] context_offset Offset from the communicator context ID - * \param[in] is_blocking Is this a blocking request? (i.e. one of MPI_Send or MPI_Ssend) - * \param[in] is_sync Is this a synchronous request? (i.e. of of MPI_Ssend or MPI_Issend) - * \param[out] request Return a pointer to the new request object - * - * \returns An MPI Error code - */ -static inline int -MPIDI_Send(const void * buf, - MPI_Aint count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - unsigned is_blocking, - unsigned is_sync, - MPIR_Request ** request) -{ - MPIR_Request * sreq = NULL; - - /* --------------------- */ - /* create a send request */ - /* --------------------- */ - - sreq = MPIDI_Request_create2(); - *request = sreq; - - /* match info */ - MPIDI_Request_setMatch(sreq, tag, comm->rank, comm->context_id+context_offset); - - /* data buffer info */ - sreq->mpid.userbuf = (void*)buf; - sreq->mpid.userbufcount = count; - sreq->mpid.datatype = datatype; - - /* communicator & destination info */ - sreq->comm = comm; MPIR_Comm_add_ref(comm); - MPIDI_Request_setPeerRank_comm(sreq, rank); - - /* message type info */ - sreq->kind = MPIR_REQUEST_KIND__SEND; - /* ----------------------------------------- */ - /* start the message */ - /* ----------------------------------------- */ - if (is_sync) - { - MPIDI_Request_uncomplete(sreq); - MPIDI_Send_post(MPIDI_Ssend_handoff, sreq); - } - else - { - MPIDI_Send_post(MPIDI_Send_handoff, sreq); - } - - return MPI_SUCCESS; -} - - - -#endif diff --git a/src/mpid/pamid/src/pt2pt/mpidi_sendmsg.c b/src/mpid/pamid/src/pt2pt/mpidi_sendmsg.c deleted file mode 100644 index a26b6fd..0000000 --- a/src/mpid/pamid/src/pt2pt/mpidi_sendmsg.c +++ /dev/null @@ -1,743 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/mpidi_sendmsg.c - * \brief Funnel point for starting all MPI messages - */ -#include <mpidimpl.h> -#ifndef min -#define min(a,b) (((a) < (b)) ? (a) : (b)) -#endif - -#if TOKEN_FLOW_CONTROL -#define MPIDI_Piggy_back_tokens MPIDI_Piggy_back_tokens_inline -static inline void * -MPIDI_Piggy_back_tokens_inline(int dest,MPIR_Request *shd,size_t len) - { - int rettoks=0; - if (MPIDI_Token_cntr[dest].rettoks) - { - rettoks=min(MPIDI_Token_cntr[dest].rettoks, TOKENS_BITMASK); - MPIDI_Token_cntr[dest].rettoks -= rettoks; - shd->mpid.envelope.msginfo.tokens = rettoks; - } - } -#endif - -static inline void -MPIDI_SendMsg_short(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest, - void * sndbuf, - unsigned sndlen, - unsigned isSync) -{ - MPIDI_MsgInfo * msginfo = &sreq->mpid.envelope.msginfo; - - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_Short, - .dest = dest, - .header = { - .iov_base = msginfo, - .iov_len = sizeof(MPIDI_MsgInfo), - }, - .data = { - .iov_base = sndbuf, - .iov_len = sndlen, - }, - }; - if (isSync) - params.dispatch = MPIDI_Protocols_ShortSync; - - pami_result_t rc; - rc = PAMI_Send_immediate(context, ¶ms); -#ifdef TRACE_ON - if (rc) - { - TRACE_ERR("sizeof(msginfo)=%zu sizeof(data)=%u\n", sizeof(MPIDI_MsgInfo), sndlen); - } -#endif - MPID_assert(rc == PAMI_SUCCESS); - TRACE_SET_S_VAL(dest,(sreq->mpid.envelope.msginfo.MPIseqno & SEQMASK),mode,params.dispatch); - if (!isSync) { - TRACE_SET_S_BIT(dest,(sreq->mpid.envelope.msginfo.MPIseqno & SEQMASK),fl.f.NoComp); - TRACE_SET_S_BIT(dest,(sreq->mpid.envelope.msginfo.MPIseqno & SEQMASK),fl.f.sendShort); - } else { - TRACE_SET_S_BIT(dest,(sreq->mpid.envelope.msginfo.MPIseqno & SEQMASK),fl.f.sendEnvelop); - } - - MPIDI_SendDoneCB_inline(context, sreq, PAMI_SUCCESS); -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->sendsComplete); -#endif -} - -static void -MPIDI_SendMsg_eager(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest, - void * sndbuf, - unsigned sndlen) - __attribute__((__noinline__)); -static void -MPIDI_SendMsg_eager(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest, - void * sndbuf, - unsigned sndlen) -{ - MPIDI_MsgInfo * msginfo = &sreq->mpid.envelope.msginfo; - - pami_send_t params = { - .send = { - .dispatch = MPIDI_Protocols_Eager, - .dest = dest, - .header = { - .iov_base = msginfo, - .iov_len = sizeof(MPIDI_MsgInfo), - }, - .data = { - .iov_base = sndbuf, - .iov_len = sndlen, - }, - }, - .events = { - .cookie = sreq, - .local_fn = MPIDI_SendDoneCB, - .remote_fn= NULL, - }, - }; - - pami_result_t rc; - rc = PAMI_Send(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),mode,MPIDI_Protocols_Eager); - TRACE_SET_S_BIT(dest,(sreq->mpid.idx),fl.f.sendEager); -} - - -static void -MPIDI_SendMsg_rzv(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest, - void * sndbuf, - size_t sndlen) - __attribute__((__noinline__)); -static void -MPIDI_SendMsg_rzv(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest, - void * sndbuf, - size_t sndlen) -{ - pami_result_t rc; - - /* Set the isRzv bit in the SEND request. This is important for - * canceling requests. - */ - MPIDI_Request_setRzv(sreq, 1); - - /* The rendezvous information, such as the origin/local/sender - * node's send buffer and the number of bytes the origin node wishes - * to send, is sent as the payload of the request-to-send (RTS) - * message. - */ -#ifdef USE_PAMI_RDMA - size_t sndlen_out; - rc = PAMI_Memregion_create(context, - sndbuf, - sndlen, - &sndlen_out, - &sreq->mpid.envelope.memregion); - MPID_assert(rc == PAMI_SUCCESS); - MPID_assert(sndlen == sndlen_out); - TRACE_ERR("RZV send for mr=%#llx addr=%p *addr[0]=%#016llx *addr[1]=%#016llx bytes=%zu\n", - *(unsigned long long*)&sreq->mpid.envelope.memregion, - sndbuf, - *(((unsigned long long*)sndbuf)+0), - *(((unsigned long long*)sndbuf)+1), - sndlen); -#else - sreq->mpid.envelope.memregion_used = 0; -#ifdef OUT_OF_ORDER_HANDLING - if ((!MPIDI_Process.mp_s_use_pami_get) && (!sreq->mpid.envelope.msginfo.noRDMA)) -#else - if (!MPIDI_Process.mp_s_use_pami_get) -#endif - { - size_t sndlen_out; - rc = PAMI_Memregion_create(context, - sndbuf, - sndlen, - &sndlen_out, - &sreq->mpid.envelope.memregion); - if(rc == PAMI_SUCCESS) - { - MPID_assert(sndlen == sndlen_out); - TRACE_ERR("RZV send for mr=%#llx addr=%p *addr[0]=%#016llx *addr[1]=%#016llx bytes=%zu\n", - *(unsigned long long*)&sreq->mpid.envelope.memregion, - sndbuf, - *(((unsigned long long*)sndbuf)+0), - *(((unsigned long long*)sndbuf)+1), - sndlen); - sreq->mpid.envelope.memregion_used = 1; - } - sreq->mpid.envelope.data = sndbuf; - } - else - { - TRACE_ERR("RZV send (failed registration for sreq=%p addr=%p *addr[0]=%#016llx *addr[1]=%#016llx bytes=%zu\n", - sreq,sndbuf, - *(((unsigned long long*)sndbuf)+0), - *(((unsigned long long*)sndbuf)+1), - sndlen); - sreq->mpid.envelope.data = sndbuf; - } -#endif - sreq->mpid.envelope.length = sndlen; - - /* Do not specify a callback function to be invoked when the RTS - * message has been sent. The MPI_Send is completed only when the - * target/remote/receiver node has completed an PAMI_Get from the - * origin node and has then sent a rendezvous acknowledgement (ACK) - * to the origin node to signify the end of the transfer. When the - * ACK message is received by the origin node the same callback - * function is used to complete the MPI_Send as the non-rendezvous - * case. - */ - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_RVZ, - .dest = dest, - .header = { - .iov_base = &sreq->mpid.envelope, - .iov_len = sizeof(MPIDI_MsgEnvelope), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - }; - - rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),bufaddr,sreq->mpid.envelope.data); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),mode,MPIDI_Protocols_RVZ); - TRACE_SET_S_BIT(dest,(sreq->mpid.idx),fl.f.sendRzv); - TRACE_SET_S_BIT(dest,(sreq->mpid.idx),fl.f.sendEnvelop); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),fl.f.memRegion,sreq->mpid.envelope.memregion_used); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),fl.f.use_pami_get,MPIDI_Process.mp_s_use_pami_get); -} - - -static void -MPIDI_SendMsg_rzv_zerobyte(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest) - __attribute__((__noinline__)); -static void -MPIDI_SendMsg_rzv_zerobyte(pami_context_t context, - MPIR_Request * sreq, - pami_endpoint_t dest) -{ - pami_result_t rc; - - /* Set the isRzv bit in the SEND request. This is important for - * canceling requests. - */ - MPIDI_Request_setRzv(sreq, 1); - - /* The rendezvous information, such as the origin/local/sender - * node's send buffer and the number of bytes the origin node wishes - * to send, is sent as the payload of the request-to-send (RTS) - * message. - */ - - sreq->mpid.envelope.data = NULL; - sreq->mpid.envelope.length = 0; - - /* Do not specify a callback function to be invoked when the RTS - * message has been sent. The MPI_Send is completed only when the - * target/remote/receiver node has matched the receive and has then - * sent a rendezvous acknowledgement (ACK) to the origin node to - * signify the end of the transfer. When the ACK message is received - * by the origin node the same callback function is used to complete - * the MPI_Send as the non-rendezvous case. - */ - pami_send_immediate_t params = { - .dispatch = MPIDI_Protocols_RVZ_zerobyte, - .dest = dest, - .header = { - .iov_base = &sreq->mpid.envelope, - .iov_len = sizeof(MPIDI_MsgEnvelope), - }, - .data = { - .iov_base = NULL, - .iov_len = 0, - }, - }; - - rc = PAMI_Send_immediate(context, ¶ms); - MPID_assert(rc == PAMI_SUCCESS); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),bufaddr,sreq->mpid.envelope.data); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),mode,MPIDI_Protocols_RVZ_zerobyte); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),fl.f.memRegion,sreq->mpid.envelope.memregion_used); - TRACE_SET_S_VAL(dest,(sreq->mpid.idx),fl.f.use_pami_get,MPIDI_Process.mp_s_use_pami_get); - TRACE_SET_S_BIT(dest,(sreq->mpid.idx),fl.f.sendRzv); - TRACE_SET_S_BIT(dest,(sreq->mpid.idx),fl.f.sendEnvelop); -} - - - -static void -MPIDI_SendMsg_process_userdefined_dt(MPIR_Request * sreq, - void ** sndbuf, - size_t * data_sz) - __attribute__((__noinline__)); -static void -MPIDI_SendMsg_process_userdefined_dt(MPIR_Request * sreq, - void ** _sndbuf, - size_t * _data_sz) -{ - size_t data_sz; - int dt_contig; - MPI_Aint dt_true_lb; - MPIDU_Datatype* dt_ptr; - void * sndbuf; - - /* - * Get the datatype info - */ - MPIDI_Datatype_get_info(sreq->mpid.userbufcount, - sreq->mpid.datatype, - dt_contig, - data_sz, - dt_ptr, - dt_true_lb); - - MPID_assert(sreq->mpid.uebuf == NULL); - - /* - * Contiguous data type - */ - if (likely(dt_contig)) - { - sndbuf = sreq->mpid.userbuf + dt_true_lb; - } - - /* - * Non-contiguous data type; allocate and populate temporary send - * buffer - */ - else - { - char *buf = NULL; -#if CUDA_AWARE_SUPPORT - // This will need to be done in two steps: - // 1 - Allocate a temp buffer which is the same size as user buffer and copy in it. - // 2 - Pack data into ue buffer from temp buffer. - int on_device = MPIDI_cuda_is_device_buf(sreq->mpid.userbuf); - if(MPIDI_Process.cuda_aware_support_on && on_device) - { - MPI_Aint dt_extent; - MPIDU_Datatype_get_extent_macro(sreq->mpid.datatype, dt_extent); - buf = MPL_malloc(dt_extent * sreq->mpid.userbufcount); - - cudaError_t cudaerr = CudaMemcpy(buf, sreq->mpid.userbuf, dt_extent * sreq->mpid.userbufcount, cudaMemcpyDeviceToHost); - if (cudaSuccess != cudaerr) { - fprintf(stderr, "cudaMalloc failed: %s\n", CudaGetErrorString(cudaerr)); - } - - } -#endif - - MPIDU_Segment segment; - - if(data_sz != 0) { - sreq->mpid.uebuf = sndbuf = MPL_malloc(data_sz); - if (unlikely(sndbuf == NULL)) - { - sreq->status.MPI_ERROR = MPI_ERR_NO_SPACE; - MPIR_STATUS_SET_COUNT(sreq->status, 0); - MPID_Abort(NULL, MPI_ERR_NO_SPACE, -1, - "Unable to allocate non-contiguous buffer"); - } - sreq->mpid.uebuf_malloc = mpiuMalloc; - - DLOOP_Offset last = data_sz; -#if CUDA_AWARE_SUPPORT - if(!MPIDI_Process.cuda_aware_support_on || !on_device) - buf = sreq->mpid.userbuf; -#else - buf = sreq->mpid.userbuf; -#endif - MPID_assert(buf != NULL); - - MPIDU_Segment_init(buf, - sreq->mpid.userbufcount, - sreq->mpid.datatype, - &segment, - 0); - MPIDU_Segment_pack(&segment, 0, &last, sndbuf); - MPID_assert(last == data_sz); -#if CUDA_AWARE_SUPPORT - if(MPIDI_Process.cuda_aware_support_on && on_device) - MPL_free(buf); -#endif - } else { - sndbuf = NULL; - } - } - - *_sndbuf = sndbuf; - *_data_sz = data_sz; -} - - -static inline void -MPIDI_SendMsg(pami_context_t context, - MPIR_Request * sreq, - unsigned isSync, - const unsigned isInternal) -{ - /* ------------------------------ */ - /* special case: NULL destination */ - /* ------------------------------ */ - int rank = MPIDI_Request_getPeerRank_comm(sreq); - if (unlikely(rank == MPI_PROC_NULL)) - { - if (isSync) - MPIDI_Request_complete(sreq); - MPIDI_Request_complete(sreq); - return; - } - else - { - MPIDI_Request_setPeerRank_pami(sreq, MPID_VCR_GET_LPID(sreq->comm->vcr, rank)); - } - - MPIDI_Request_setSync(sreq, isSync); - MPIDI_Request_setPeerRequestH(sreq); - - /* - * Create the destination endpoint - */ - pami_endpoint_t dest; - MPIDI_Context_endpoint(sreq, &dest); - pami_task_t dest_tid; - dest_tid=sreq->comm->vcr[rank]->taskid; -#if (MPIDI_STATISTICS) - MPID_NSTAT(mpid_statp->sends); -#endif -#ifdef OUT_OF_ORDER_HANDLING - MPIDI_Out_cntr_t *out_cntr; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - out_cntr = &MPIDI_Out_cntr[dest_tid]; - out_cntr->nMsgs++; - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIDI_Request_setMatchSeq(sreq, out_cntr->nMsgs); -#endif -if (!TOKEN_FLOW_CONTROL_ON) { - size_t data_sz; - void * sndbuf; - if (likely(HANDLE_GET_KIND(sreq->mpid.datatype) == HANDLE_KIND_BUILTIN)) - { - sndbuf = sreq->mpid.userbuf; - data_sz = sreq->mpid.userbufcount * MPIDU_Datatype_get_basic_size(sreq->mpid.datatype); - } - else - { - MPIDI_SendMsg_process_userdefined_dt(sreq, &sndbuf, &data_sz); - } - MPIDI_GET_S_REC(dest_tid,sreq,context,isSync,data_sz); - -#ifdef OUT_OF_ORDER_HANDLING - sreq->mpid.envelope.msginfo.noRDMA=0; -#endif - - const unsigned isLocal = PAMIX_Task_is_local(dest_tid); - const size_t data_sz_limit = isSync?ULONG_MAX:data_sz; - - /* - * Always use the short protocol when data_sz is small. - */ - if (likely(data_sz < MPIDI_PT2PT_SHORT_LIMIT(isInternal,isLocal))) - { - TRACE_ERR("Sending(short%s%s) bytes=%zu (short_limit=%u)\n", isInternal==1?",internal":"", isLocal==1?",intranode":"", data_sz, MPIDI_PT2PT_SHORT_LIMIT(isInternal,isLocal)); - MPIDI_SendMsg_short(context, - sreq, - dest, - sndbuf, - data_sz, - isSync); - } - /* - * Use the eager protocol when data_sz is less than the eager limit. - */ - else if (data_sz_limit < MPIDI_PT2PT_EAGER_LIMIT(isInternal,isLocal)) - { - TRACE_ERR("Sending(eager%s%s) bytes=%zu (eager_limit=%u)\n", isInternal==1?",internal":"", isLocal==1?",intranode":"", data_sz, MPIDI_PT2PT_EAGER_LIMIT(isInternal,isLocal)); - MPIDI_SendMsg_eager(context, - sreq, - dest, - sndbuf, - data_sz); -#ifdef MPIDI_STATISTICS - if (!isLocal && MPIR_cc_is_complete(&sreq->cc)) - { - MPID_NSTAT(mpid_statp->sendsComplete); - } -#endif - } - /* - * Use the default rendezvous protocol implementation that guarantees - * no unexpected data and does not complete the send until the remote - * receive is posted. - */ - else - { - TRACE_ERR("Sending(rendezvous%s%s) bytes=%zu (eager_limit=%u)\n", isInternal==1?",internal":"", isLocal==1?",intranode":"", data_sz, MPIDI_PT2PT_EAGER_LIMIT(isInternal,isLocal)); -#ifdef OUT_OF_ORDER_HANDLING - sreq->mpid.envelope.msginfo.noRDMA=isLocal; -#endif - if (likely(data_sz > 0)) - { - MPIDI_SendMsg_rzv(context, - sreq, - dest, - sndbuf, - data_sz); - } - else - { - MPIDI_SendMsg_rzv_zerobyte(context, sreq, dest); - } - -#ifdef MPIDI_STATISTICS - if (!isLocal && MPIR_cc_is_complete(&sreq->cc)) - { - MPID_NSTAT(mpid_statp->sendsComplete); - } -#endif - } - } - else - { /* TOKEN_FLOW_CONTROL_ON */ - #if TOKEN_FLOW_CONTROL - if (!(sreq->mpid.userbufcount)) - { - MPIDI_GET_S_REC(dest_tid,sreq,context,isSync,0); - TRACE_ERR("Sending(short,intranode) bytes=%u (short_limit=%u)\n", data_sz, MPIDI_Process.short_limit); - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - MPIDI_Piggy_back_tokens(dest,sreq,0); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIDI_SendMsg_short(context, - sreq, - dest, - sreq->mpid.userbuf, - 0, - isSync); - } - else - { - size_t data_sz; - void * sndbuf; - int noRDMA=0; - if (likely(HANDLE_GET_KIND(sreq->mpid.datatype) == HANDLE_KIND_BUILTIN)) - { - sndbuf = sreq->mpid.userbuf; - data_sz = sreq->mpid.userbufcount * MPIDU_Datatype_get_basic_size(sreq->mpid.datatype); - } - else - { - MPIDI_SendMsg_process_userdefined_dt(sreq, &sndbuf, &data_sz); - } - MPIDI_GET_S_REC(dest_tid,sreq,context,isSync,data_sz); - if (unlikely(PAMIX_Task_is_local(dest_tid) != 0)) noRDMA=1; - - MPIU_THREAD_CS_ENTER(MSGQUEUE,0); - if ((!isSync) && MPIDI_Token_cntr[dest].tokens >= 1) - { - if (data_sz <= MPIDI_Process.pt2pt.limits.application.immediate.remote) - { - TRACE_ERR("Sending(short,intranode) bytes=%u (short_limit=%u)\n", data_sz,MPIDI_Process.pt2pt.limits.application.immediate.remote); - --MPIDI_Token_cntr[dest].tokens; - MPIDI_Piggy_back_tokens(dest,sreq,data_sz); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIDI_SendMsg_short(context, - sreq, - dest, - sndbuf, - data_sz, - isSync); - } - else if (data_sz <= MPIDI_Process.pt2pt.limits.application.eager.remote) - { - TRACE_ERR("Sending(eager) bytes=%u (eager_limit=%u)\n", data_sz, MPIDI_Process.pt2pt.limits.application.eager.remote); - --MPIDI_Token_cntr[dest].tokens; - MPIDI_Piggy_back_tokens(dest,sreq,data_sz); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - MPIDI_SendMsg_eager(context, - sreq, - dest, - sndbuf, - data_sz); -#ifdef MPIDI_STATISTICS - if (MPIR_cc_is_complete(&sreq->cc)) { - MPID_NSTAT(mpid_statp->sendsComplete); - } -#endif - - } - else /* rendezvous message */ - { - TRACE_ERR("Sending(RZV) bytes=%u (eager_limit=%u)\n", data_sz, MPIDI_Process.pt2pt.limits.application.eager.remote); - MPIDI_Piggy_back_tokens(dest,sreq,data_sz); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - sreq->mpid.envelope.msginfo.noRDMA=noRDMA; - MPIDI_SendMsg_rzv(context, - sreq, - dest, - sndbuf, - data_sz); -#ifdef MPIDI_STATISTICS - if (MPIR_cc_is_complete(&sreq->cc)) - { - MPID_NSTAT(mpid_statp->sendsComplete); - } -#endif - } - } - else - { /* no tokens, all messages use rendezvous protocol */ - if ((data_sz <= MPIDI_Process.pt2pt.limits.application.eager.remote) && (!isSync)) { - ++MPIDI_Token_cntr[dest].n_tokenStarved; - sreq->mpid.envelope.msginfo.noRDMA=1; - } - else sreq->mpid.envelope.msginfo.noRDMA=noRDMA; - MPIDI_Piggy_back_tokens(dest,sreq,data_sz); - MPIU_THREAD_CS_EXIT(MSGQUEUE,0); - TRACE_ERR("Sending(RZV) bytes=%u (eager_limit=%u)\n", data_sz, MPIDI_Process.pt2pt.limits.application.eager.remote); - if (likely(data_sz > 0)) - { - MPIDI_SendMsg_rzv(context, - sreq, - dest, - sndbuf, - data_sz); - } - else - { - MPIDI_SendMsg_rzv_zerobyte(context, sreq, dest); - } -#ifdef MPIDI_STATISTICS - if (MPIR_cc_is_complete(&sreq->cc)) - { - MPID_NSTAT(mpid_statp->sendsComplete); - } -#endif - } - } - #else - MPID_assert_always(0); - #endif /* TOKEN_FLOW_CONTROL */ - } -} - - -/* - * \brief Central function for all low-level sends. - * - * This is assumed to have been posted to a context, and is now being - * called from inside advance. This has (unspecified) locking - * implications. - * - * Prerequisites: - * + Not sending to a NULL rank - * + Request already allocated - * - * \param[in] context The PAMI context on which to do the send operation - * \param[in,out] sreq Structure containing all relevant info about the message. - */ -pami_result_t -MPIDI_Send_handoff(pami_context_t context, - void * _sreq) -{ - MPIR_Request * sreq = (MPIR_Request*)_sreq; - MPID_assert(sreq != NULL); - - MPIDI_SendMsg(context, sreq, 0, 0); - return PAMI_SUCCESS; -} - - -pami_result_t -MPIDI_Ssend_handoff(pami_context_t context, - void * _sreq) -{ - MPIR_Request * sreq = (MPIR_Request*)_sreq; - MPID_assert(sreq != NULL); - - MPIDI_SendMsg(context, sreq, 1, 0); - return PAMI_SUCCESS; -} - - -/* - * \brief Central function for all low-level sends. - * - * This is assumed to have been posted to a context, and is now being - * called from inside advance. This has (unspecified) locking - * implications. - * - * Prerequisites: - * + Not sending to a NULL rank - * + Request already allocated - * - * \param[in] context The PAMI context on which to do the send operation - * \param[in,out] sreq Structure containing all relevant info about the message. - */ -pami_result_t -MPIDI_Isend_handoff(pami_context_t context, - void * _sreq) -{ - MPIR_Request * sreq = (MPIR_Request*)_sreq; - MPID_assert(sreq != NULL); - - /* This initializes all the fields not set in MPI_Isend() */ - MPIDI_Request_initialize(sreq); - - /* Since this is only called from MPI_Isend(), it is not synchronous */ - MPIDI_SendMsg(context, sreq, 0, 0); - return PAMI_SUCCESS; -} - -pami_result_t -MPIDI_Isend_handoff_internal(pami_context_t context, - void * _sreq) -{ - MPIR_Request * sreq = (MPIR_Request*)_sreq; - MPID_assert(sreq != NULL); - - /* This initializes all the fields not set in MPI_Isend() */ - MPIDI_Request_initialize(sreq); - - /* Since this is only called from MPI_Isend(), it is not synchronous */ - MPIDI_SendMsg(context, sreq, 0, 1); - return PAMI_SUCCESS; -} diff --git a/src/mpid/pamid/src/pt2pt/persistent/Makefile.mk b/src/mpid/pamid/src/pt2pt/persistent/Makefile.mk deleted file mode 100644 index e2894a6..0000000 --- a/src/mpid/pamid/src/pt2pt/persistent/Makefile.mk +++ /dev/null @@ -1,30 +0,0 @@ -# begin_generated_IBM_copyright_prolog -# -# This is an automatically generated copyright prolog. -# After initializing, DO NOT MODIFY OR MOVE -# --------------------------------------------------------------- -# Licensed Materials - Property of IBM -# Blue Gene/Q 5765-PER 5765-PRP -# -# (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -# US Government Users Restricted Rights - -# Use, duplication, or disclosure restricted -# by GSA ADP Schedule Contract with IBM Corp. -# -# --------------------------------------------------------------- -# -# end_generated_IBM_copyright_prolog -# -*- mode: makefile-gmake; -*- - -# note that the includes always happen but the effects of their contents are -# affected by "if BUILD_PAMID" -if BUILD_PAMID - - -mpi_core_sources += \ - src/mpid/pamid/src/pt2pt/persistent/mpid_recv_init.c \ - src/mpid/pamid/src/pt2pt/persistent/mpid_send_init.c \ - src/mpid/pamid/src/pt2pt/persistent/mpid_startall.c - -endif BUILD_PAMID - diff --git a/src/mpid/pamid/src/pt2pt/persistent/mpid_recv_init.c b/src/mpid/pamid/src/pt2pt/persistent/mpid_recv_init.c deleted file mode 100644 index 63ebbeb..0000000 --- a/src/mpid/pamid/src/pt2pt/persistent/mpid_recv_init.c +++ /dev/null @@ -1,56 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/persistent/mpid_recv_init.c - * \brief ??? - */ - -/* This creates and initializes a persistent recv request */ - -#include <mpidimpl.h> - -int MPID_Recv_init(void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - MPIR_Request * rreq = *request = MPIDI_Request_create2(); - - rreq->kind = MPIR_REQUEST_KIND__PREQUEST_RECV; - rreq->comm = comm; - MPIR_Comm_add_ref(comm); - MPIDI_Request_setMatch(rreq, tag, rank, comm->recvcontext_id+context_offset); - rreq->mpid.userbuf = buf; - rreq->mpid.userbufcount = count; - rreq->mpid.datatype = datatype; - rreq->partner_request = NULL; - MPIDI_Request_complete(rreq); - - MPIDI_Request_setPType(rreq, MPIDI_REQUEST_PTYPE_RECV); - if (HANDLE_GET_KIND(datatype) != HANDLE_KIND_BUILTIN) - { - MPIDU_Datatype_get_ptr(datatype, rreq->mpid.datatype_ptr); - MPIDU_Datatype_add_ref(rreq->mpid.datatype_ptr); - } - - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/pt2pt/persistent/mpid_send_init.c b/src/mpid/pamid/src/pt2pt/persistent/mpid_send_init.c deleted file mode 100644 index 35080e5..0000000 --- a/src/mpid/pamid/src/pt2pt/persistent/mpid_send_init.c +++ /dev/null @@ -1,148 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/persistent/mpid_send_init.c - * \brief ??? - */ -/* This creates and initializes a persistent send request */ - -#include <mpidimpl.h> - -/** - * *************************************************************************** - * create a persistent send template - * *************************************************************************** - */ - -static inline int -MPID_PSendRequest(const void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - MPIR_Request* sreq = *request = MPIDI_Request_create2(); - - sreq->kind = MPIR_REQUEST_KIND__PREQUEST_SEND; - sreq->comm = comm; - MPIR_Comm_add_ref(comm); - MPIDI_Request_setMatch(sreq, tag, rank, comm->context_id+context_offset); - sreq->mpid.userbuf = (void*)buf; - sreq->mpid.userbufcount = count; - sreq->mpid.datatype = datatype; - sreq->partner_request = NULL; - MPIDI_Request_complete(sreq); - - if (HANDLE_GET_KIND(datatype) != HANDLE_KIND_BUILTIN) - { - MPIDU_Datatype_get_ptr(datatype, sreq->mpid.datatype_ptr); - MPIDU_Datatype_add_ref(sreq->mpid.datatype_ptr); - } - - return MPI_SUCCESS; -} - -/** - * *************************************************************************** - * simple persistent send - * *************************************************************************** - */ - -int MPID_Send_init(const void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - int mpi_errno = MPID_PSendRequest(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - request); - if (mpi_errno != MPI_SUCCESS) - return mpi_errno; - MPIDI_Request_setPType((*request), MPIDI_REQUEST_PTYPE_SEND); - return MPI_SUCCESS; -} - -/** - * *************************************************************************** - * persistent synchronous send - * *************************************************************************** - */ - -int MPID_Ssend_init(const void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - int mpi_errno = MPID_PSendRequest(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - request); - if (mpi_errno != MPI_SUCCESS) - return mpi_errno; - MPIDI_Request_setPType((*request), MPIDI_REQUEST_PTYPE_SSEND); - return MPI_SUCCESS; -} - -/** - * *************************************************************************** - * persistent buffered send - * *************************************************************************** - */ - -int MPID_Bsend_init(const void * buf, - int count, - MPI_Datatype datatype, - int rank, - int tag, - MPIR_Comm * comm, - int context_offset, - MPIR_Request ** request) -{ - int mpi_errno = MPID_PSendRequest(buf, - count, - datatype, - rank, - tag, - comm, - context_offset, - request); - if (mpi_errno != MPI_SUCCESS) - return mpi_errno; - MPIDI_Request_setPType((*request), MPIDI_REQUEST_PTYPE_BSEND); - return MPI_SUCCESS; -} diff --git a/src/mpid/pamid/src/pt2pt/persistent/mpid_startall.c b/src/mpid/pamid/src/pt2pt/persistent/mpid_startall.c deleted file mode 100644 index dbb1c3a..0000000 --- a/src/mpid/pamid/src/pt2pt/persistent/mpid_startall.c +++ /dev/null @@ -1,140 +0,0 @@ -/* begin_generated_IBM_copyright_prolog */ -/* */ -/* This is an automatically generated copyright prolog. */ -/* After initializing, DO NOT MODIFY OR MOVE */ -/* --------------------------------------------------------------- */ -/* Licensed Materials - Property of IBM */ -/* Blue Gene/Q 5765-PER 5765-PRP */ -/* */ -/* (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved */ -/* US Government Users Restricted Rights - */ -/* Use, duplication, or disclosure restricted */ -/* by GSA ADP Schedule Contract with IBM Corp. */ -/* */ -/* --------------------------------------------------------------- */ -/* */ -/* end_generated_IBM_copyright_prolog */ -/* (C)Copyright IBM Corp. 2007, 2011 */ -/** - * \file src/pt2pt/persistent/mpid_startall.c - * \brief ??? - */ -#include <mpidimpl.h> -#include <../mpi/pt2pt/bsendutil.h> - -int MPID_Startall(int count, MPIR_Request * requests[]) -{ - int rc=MPI_SUCCESS, i; - for (i = 0; i < count; i++) - { - MPIR_Request * const preq = requests[i]; - switch(MPIDI_Request_getPType(preq)) - { - case MPIDI_REQUEST_PTYPE_RECV: - { - rc = MPID_Irecv(preq->mpid.userbuf, - preq->mpid.userbufcount, - preq->mpid.datatype, - MPIDI_Request_getMatchRank(preq), - MPIDI_Request_getMatchTag(preq), - preq->comm, - MPIDI_Request_getMatchCtxt(preq) - preq->comm->recvcontext_id, - &preq->partner_request); - break; - } - case MPIDI_REQUEST_PTYPE_SEND: - { - rc = MPID_Isend(preq->mpid.userbuf, - preq->mpid.userbufcount, - preq->mpid.datatype, - MPIDI_Request_getMatchRank(preq), - MPIDI_Request_getMatchTag(preq), - preq->comm, - MPIDI_Request_getMatchCtxt(preq) - preq->comm->context_id, - &preq->partner_request); - break; - } - case MPIDI_REQUEST_PTYPE_SSEND: - { - rc = MPID_Issend(preq->mpid.userbuf, - preq->mpid.userbufcount, - preq->mpid.datatype, - MPIDI_Request_getMatchRank(preq), - MPIDI_Request_getMatchTag(preq), - preq->comm, - MPIDI_Request_getMatchCtxt(preq) - preq->comm->context_id, - &preq->partner_request); - break; - } - case MPIDI_REQUEST_PTYPE_BSEND: - { - rc = MPIR_Bsend_isend(preq->mpid.userbuf, - preq->mpid.userbufcount, - preq->mpid.datatype, - MPIDI_Request_getMatchRank(preq), - MPIDI_Request_getMatchTag(preq), - preq->comm, - BSEND_INIT, - &preq->partner_request); - /* - * MPICH maintains an independant reference to the child, - * but doesn't refcount it. Since they actually call - * MPI_Test() on the child request (which will release a - * ref iff the request is complete), we have to increment - * the ref_count so that it doesn't get freed from under - * us. - */ - if (preq->partner_request != NULL) - MPIR_Object_add_ref(preq->partner_request); - break; - } - - default: - { - rc = MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_FATAL, - __FUNCTION__, - __LINE__, - MPI_ERR_INTERN, - "**ch3|badreqtype", - "**ch3|badreqtype %d", - MPIDI_Request_getPType(preq)); - } - - } /* switch should end here, bug fixed. */ - - if (rc == MPI_SUCCESS) - { - preq->status.MPI_ERROR = MPI_SUCCESS; - if (MPIDI_Request_getPType(preq) == MPIDI_REQUEST_PTYPE_BSEND) - { - /* - * Complete a persistent Bsend immediately. - * - * Because the child of a persistent Bsend is just a - * normal Isend on a temp buffer, we don't need to wait on - * the child when the user calls MPI_Wait on the parent. - * Therefore, disconnect the cc_ptr link to the child and - * mark the parent complete. - */ - preq->cc_ptr = &preq->cc; - MPID_Request_set_completed(preq); - } - else - preq->cc_ptr = &preq->partner_request->cc; - } - else - { - /* If a failure occurs attempting to start the request, - then we assume that partner request was not created, - and stuff the error code in the persistent request. - The wait and test routines will look at the error code - in the persistent request if a partner request is not present. */ - preq->partner_request = NULL; - preq->status.MPI_ERROR = rc; - preq->cc_ptr = &preq->cc; - MPID_Request_set_completed(preq); - } - } /* for */ - return rc; -} diff --git a/src/mpid/pamid/subconfigure.m4 b/src/mpid/pamid/subconfigure.m4 deleted file mode 100644 index 817b240..0000000 --- a/src/mpid/pamid/subconfigure.m4 +++ /dev/null @@ -1,351 +0,0 @@ -[#] start of __file__ -dnl begin_generated_IBM_copyright_prolog -dnl -dnl This is an automatically generated copyright prolog. -dnl After initializing, DO NOT MODIFY OR MOVE -dnl --------------------------------------------------------------- -dnl Licensed Materials - Property of IBM -dnl Blue Gene/Q 5765-PER 5765-PRP -dnl -dnl (C) Copyright IBM Corp. 2011, 2012 All Rights Reserved -dnl US Government Users Restricted Rights - -dnl Use, duplication, or disclosure restricted -dnl by GSA ADP Schedule Contract with IBM Corp. -dnl -dnl --------------------------------------------------------------- -dnl -dnl end_generated_IBM_copyright_prolog -dnl -*- mode: makefile-gmake; -*- - -dnl MPICH_SUBCFG_BEFORE=src/mpid/common/sched -dnl MPICH_SUBCFG_BEFORE=src/mpid/common/datatype -dnl MPICH_SUBCFG_BEFORE=src/mpid/common/thread -dnl MPICH_SUBCFG_BEFORE=src/mpid/common/shm - -dnl _PREREQ handles the former role of mpichprereq, setup_device, etc -[#] expansion is: PAC_SUBCFG_PREREQ_[]PAC_SUBCFG_AUTO_SUFFIX -AC_DEFUN([PAC_SUBCFG_PREREQ_]PAC_SUBCFG_AUTO_SUFFIX,[ -AM_CONDITIONAL([BUILD_PAMID],[test "$device_name" = "pamid"]) -AM_CONDITIONAL([QUEUE_BINARY_SEARCH_SUPPORT],[test "$enable_queue_binary_search" = "yes"]) - -AC_ARG_VAR([PAMILIBNAME],[can be used to override the name of the PAMI library (default: "pami")]) -original_PAMILIBNAME=${PAMILIBNAME} -PAMILIBNAME=${PAMILIBNAME:-"pami"} -AC_SUBST(PAMILIBNAME) -export PAMILIBNAME - -dnl this subconfigure.m4 handles the configure work for the ftb subdir too -dnl this AM_CONDITIONAL only works because enable_ftb is set very early on by -dnl autoconf's argument parsing code. The "action-if-given" from the -dnl AC_ARG_ENABLE has not yet run -dnl AM_CONDITIONAL([BUILD_CH3_UTIL_FTB],[test "x$enable_ftb" = "xyes"]) - -AM_COND_IF([BUILD_PAMID],[ - -pamid_platform=${device_args} -if test x"$pamid_platform" == "x"; then - AS_CASE([$host], - [*-bgq-*],[pamid_platform=BGQ]) -fi - -AC_MSG_NOTICE([Using the pamid platform '$pamid_platform']) - - -dnl Set a value for the maximum processor name. -MPID_MAX_PROCESSOR_NAME=128 -PM_REQUIRES_PMI=pmi2 -if test "${pamid_platform}" = "PE" ; then - with_shared_memory=sysv - PM_REQUIRES_PMI=pmi2/poe -elif test "${pamid_platform}" = "BGQ" ; then - with_shared_memory=mmap - MPID_DEFAULT_CROSS_FILE=${master_top_srcdir}/src/mpid/pamid/cross/bgq8 - MPID_DEFAULT_PM=no -fi - -MPID_MAX_THREAD_LEVEL=MPI_THREAD_MULTIPLE - -dnl the PAMID device depends on the common NBC scheduler SHM code -build_mpid_common_sched=yes -build_mpid_common_datatype=yes -build_mpid_common_thread=yes -build_mpid_common_shm=yes - - -])dnl end AM_COND_IF(BUILD_PAMID,...) -])dnl end PREREQ -AC_DEFUN([PAC_SUBCFG_BODY_]PAC_SUBCFG_AUTO_SUFFIX,[ -AM_COND_IF([BUILD_PAMID],[ -AC_MSG_NOTICE([RUNNING CONFIGURE FOR PAMI DEVICE]) - - -ASSERT_LEVEL=2 -AC_ARG_WITH(assert-level, - AS_HELP_STRING([--with-assert-level={0 1 2}],[pamid build assert-level (default: 2)]), - [ ASSERT_LEVEL=$withval ]) -AC_SUBST(ASSERT_LEVEL) -AC_DEFINE_UNQUOTED([ASSERT_LEVEL], $ASSERT_LEVEL, [The pamid assert level]) - -dnl -dnl This macro adds the -I to CPPFLAGS and/or the -L to LDFLAGS -dnl -PAC_SET_HEADER_LIB_PATH(pami) - -dnl -dnl Set the pamid platform define. -dnl -PAC_APPEND_FLAG([-D__${pamid_platform}__], [CPPFLAGS]) - -dnl Check for available shared memory functions -PAC_ARG_SHARED_MEMORY -if test "$with_shared_memory" != "mmap" -a "$with_shared_memory" != "sysv"; then - AC_MSG_ERROR([cannot support shared memory: need either sysv shared memory functions or mmap in order to support shared memory]) -fi - -dnl -dnl The default is to enable the use of the recv queue binary search -dnl ... except on BGQ -dnl -enable_queue_binary_search=yes -if test "${pamid_platform}" = "BGQ" ; then - enable_queue_binary_search=no -fi - -AM_CONDITIONAL([QUEUE_BINARY_SEARCH_SUPPORT],[test "$enable_queue_binary_search" = "yes"]) - -dnl -dnl This configure option allows "sandbox" bgq system software to be used. -dnl -AC_ARG_WITH(bgq-install-dir, - AS_HELP_STRING([--with-bgq-install-dir=PATH],[specify path where bgq system software can be found; - may also be specified with the 'BGQ_INSTALL_DIR' - environment variable]), - [ BGQ_INSTALL_DIR=$withval ]) -AC_SUBST(BGQ_INSTALL_DIR) - -dnl -dnl Add bgq-specific build options. -dnl -if test "${pamid_platform}" = "BGQ" ; then - - AC_MSG_CHECKING([for BGQ system software directory]) - - dnl - dnl Specify the default bgq system software paths - dnl - bgq_driver_search_path="${BGQ_INSTALL_DIR} " - for bgq_version in `echo 4 3 2 1`; do - for bgq_release in `echo 4 3 2 1`; do - for bgq_mod in `echo 4 3 2 1 0`; do - bgq_driver_search_path+="/bgsys/drivers/V${bgq_version}R${bgq_release}M${bgq_mod}/ppc64 " - done - done - done - - dnl Look for a bgq driver to use. - for bgq_driver in $bgq_driver_search_path ; do - if test -d ${bgq_driver}/spi/include ; then - - found_bgq_driver=yes - - PAC_APPEND_FLAG([-I${bgq_driver}], [CPPFLAGS]) - PAC_APPEND_FLAG([-I${bgq_driver}/spi/include/kernel/cnk], [CPPFLAGS]) - PAC_APPEND_FLAG([-L${bgq_driver}/spi/lib], [LDFLAGS]) - PAC_APPEND_FLAG([-L${bgq_driver}/spi/lib], [WRAPPER_LDFLAGS]) - - dnl If the '--with-pami' and the '--with-pami-include' configure options - dnl were NOT specified then test for a V1R2M1+ comm include directory. - dnl - AS_IF([test "x${with_pami_include}" = "x" ], - [AS_IF([test "x${with_pami}" = "x" ], - [AS_IF([test -d ${bgq_driver}/comm/include ], - [PAC_APPEND_FLAG([-I${bgq_driver}/comm/include], [CPPFLAGS])], - [PAC_APPEND_FLAG([-I${bgq_driver}/comm/sys/include],[CPPFLAGS])])])]) - - dnl If the '--with-pami' and the '--with-pami-lib' configure options were - dnl NOT specified then - dnl - dnl if a custom pami library name was NOT specified then test for a - dnl V1R2M1+ pami lib in a V1R2M1+ comm lib directory then test for a - dnl pre-V1R2M1 pami lib in a pre-V1R2M1 comm lib directory; otherwise - dnl - dnl if a custom pami library name WAS specified then test for a custom - dnl pami lib in a V1R2M1+ comm lib directory then test for a custom - dnl pami lib in a pre-V1R2M1 comm lib directory - dnl - AS_IF([test "x${with_pami_lib}" = "x" ], - [AS_IF([test "x${with_pami}" = "x" ], - [AS_IF([test "x${original_PAMILIBNAME}" = "x" ], - [AS_IF([test -f ${bgq_driver}/comm/lib/libpami-gcc.a ], - [PAMILIBNAME=pami-gcc - PAC_APPEND_FLAG([-L${bgq_driver}/comm/lib], [LDFLAGS]) - PAC_APPEND_FLAG([-L${bgq_driver}/comm/lib], [WRAPPER_LDFLAGS])], - [AS_IF([test -f ${bgq_driver}/comm/sys/lib/libpami.a ], - [PAC_APPEND_FLAG([-L${bgq_driver}/comm/sys/lib],[LDFLAGS]) - PAC_APPEND_FLAG([-L${bgq_driver}/comm/sys/lib],[WRAPPER_LDFLAGS])])])], - [AS_IF([test -f ${bgq_driver}/comm/lib/lib${PAMILIBNAME}.a ], - [PAC_APPEND_FLAG([-L${bgq_driver}/comm/lib], [LDFLAGS]) - PAC_APPEND_FLAG([-L${bgq_driver}/comm/lib], [WRAPPER_LDFLAGS])], - [AS_IF([test -f ${bgq_driver}/comm/sys/lib/lib${PAMILIBNAME}.a ], - [PAC_APPEND_FLAG([-L${bgq_driver}/comm/sys/lib],[LDFLAGS]) - PAC_APPEND_FLAG([-L${bgq_driver}/comm/sys/lib],[WRAPPER_LDFLAGS])])])])])]) - - break - fi - done - - if test "x${found_bgq_driver}" = "xyes"; then - AC_MSG_RESULT('$bgq_driver') - else - AC_MSG_RESULT('no') - fi - - dnl - dnl The bgq compile requires these libraries. - dnl - PAC_APPEND_FLAG([-lSPI], [LIBS]) - PAC_APPEND_FLAG([-lSPI_cnk], [LIBS]) - PAC_APPEND_FLAG([-lrt], [LIBS]) - PAC_APPEND_FLAG([-lpthread], [LIBS]) - PAC_APPEND_FLAG([-lstdc++], [LIBS]) - - dnl - dnl The wrapper scripts require these libraries. - dnl - PAC_APPEND_FLAG([-l${PAMILIBNAME}], [WRAPPER_LIBS]) - PAC_APPEND_FLAG([-lSPI], [WRAPPER_LIBS]) - PAC_APPEND_FLAG([-lSPI_cnk], [WRAPPER_LIBS]) - PAC_APPEND_FLAG([-lrt], [WRAPPER_LIBS]) - PAC_APPEND_FLAG([-lpthread], [WRAPPER_LIBS]) - PAC_APPEND_FLAG([-lstdc++], [WRAPPER_LIBS]) - - - AC_SEARCH_LIBS([PAMI_Send], [${PAMILIBNAME} pami-gcc]) - - - dnl For some reason, on bgq, libtool will incorrectly attempt a static link - dnl of libstdc++.so unless this '-all-static' option is used. This seems to - dnl be a problem specific to libstdc++. - dnl - dnl Only the 'cpi', 'mpivars', and 'mpichversion' executables have this problem. - MPID_LIBTOOL_STATIC_FLAG="-all-static" - - dnl Another bgq special case. The default linker behavior is to use static versions - dnl of libraries. This makes supporting interlibrary dependencies difficult. Just - dnl disable them to make our lives easier. - if test "$INTERLIB_DEPS" = "yes"; then - INTERLIB_DEPS="no" - fi -fi - -if test "${pamid_platform}" = "PE" ; then - MPID_MAX_ERROR_STRING=512 -fi - -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/include], [CPPFLAGS]) -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/util/wrappers], [CPPFLAGS]) -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/mpid/pamid/include], [CPPFLAGS]) -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/mpid/common/datatype], [CPPFLAGS]) -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/mpid/common/locks], [CPPFLAGS]) -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/mpid/common/thread], [CPPFLAGS]) -PAC_APPEND_FLAG([-I${master_top_srcdir}/src/mpid/common/sched], [CPPFLAGS]) - -dnl -dnl Check for PAMI_IN_PLACE -dnl -AC_MSG_CHECKING([for PAMI_IN_PLACE support]) -have_pami_in_place=0 -AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM([[#include "pami.h"]], - [[void * foo = PAMI_IN_PLACE;]])], - have_pami_in_place=1 -) -if test "$have_pami_in_place" != "0"; then - AC_DEFINE(HAVE_PAMI_IN_PLACE,1,[Define if PAMI_IN_PLACE is defined in pami.h]) - AC_MSG_RESULT('yes') -else - AC_DEFINE(PAMI_IN_PLACE,((void *) -1L),[Define if PAMI_IN_PLACE is not defined in pami.h]) - AC_MSG_RESULT('no') -fi - -dnl -dnl Check for PAMI_CLIENT_NONCONTIG -dnl -AC_MSG_CHECKING([for PAMI_CLIENT_NONCONTIG support]) -have_pami_client_noncontig=0 -AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM([[#include "pami.h"]], - [[int foo = PAMI_CLIENT_NONCONTIG;]])], - have_pami_client_noncontig=1 -) -if test "$have_pami_client_noncontig" != "0"; then - AC_DEFINE(HAVE_PAMI_CLIENT_NONCONTIG,1,[Define if PAMI_CLIENT_NONCONTIG is defined in pami.h]) - AC_MSG_RESULT('yes') -else - AC_MSG_RESULT('no') -fi - -dnl -dnl Check for PAMI_CLIENT_MEMORY_OPTIMIZE -dnl -AC_MSG_CHECKING([for PAMI_CLIENT_MEMORY_OPTIMIZE support]) -have_pami_client_memory_optimize=0 -AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM([[#include "pami.h"]], - [[int foo = PAMI_CLIENT_MEMORY_OPTIMIZE;]])], - have_pami_client_memory_optimize=1 -) -if test "$have_pami_client_memory_optimize" != "0"; then - AC_DEFINE(HAVE_PAMI_CLIENT_MEMORY_OPTIMIZE,1,[Define if PAMI_CLIENT_MEMORY_OPTIMIZE is defined in pami.h]) - AC_MSG_RESULT('yes') -else - AC_MSG_RESULT('no') -fi - -dnl -dnl Check for PAMI_GEOMETRY_NONCONTIG -dnl -AC_MSG_CHECKING([for PAMI_GEOMETRY_NONCONTIG support]) -have_pami_geometry_noncontig=0 -AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM([[#include "pami.h"]], - [[int foo = PAMI_GEOMETRY_NONCONTIG;]])], - have_pami_geometry_noncontig=1 -) -if test "$have_pami_geometry_noncontig" != "0"; then - AC_DEFINE(HAVE_PAMI_GEOMETRY_NONCONTIG,1,[Define if PAMI_GEOMETRY_NONCONTIG is defined in pami.h]) - AC_MSG_RESULT('yes') -else - AC_MSG_RESULT('no') -fi - -dnl -dnl Check for PAMI_GEOMETRY_MEMORY_OPTIMIZE -dnl -AC_MSG_CHECKING([for PAMI_GEOMETRY_MEMORY_OPTIMIZE support]) -have_pami_geometry_memory_optimize=0 -AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM([[#include "pami.h"]], - [[int foo = PAMI_GEOMETRY_MEMORY_OPTIMIZE;]])], - have_pami_geometry_memory_optimize=1 -) -if test "$have_pami_geometry_memory_optimize" != "0"; then - AC_DEFINE(HAVE_PAMI_GEOMETRY_MEMORY_OPTIMIZE,1,[Define if PAMI_GEOMETRY_MEMORY_OPTIMIZE is defined in pami.h]) - AC_MSG_RESULT('yes') -else - AC_MSG_RESULT('no') -fi - - - - - - - - - - -])dnl end AM_COND_IF(BUILD_PAMID,...) -])dnl end _BODY - -[#] end of __file__ diff --git a/src/mpl/configure.ac b/src/mpl/configure.ac index e07f2db..cf34469 100644 --- a/src/mpl/configure.ac +++ b/src/mpl/configure.ac @@ -234,7 +234,6 @@ dnl Unix gettimeofday (one of two versions) dnl dnl Also available are various hardware time stamps dnl Linux-x86 cycle counter -dnl BG/Q PAMI-based timer dnl dnl We also allow --enable-timer-type=name to select a timer type AC_ARG_ENABLE(timer-type, diff --git a/src/pm/hydra/mpichprereq b/src/pm/hydra/mpichprereq index bfbf778..d6aa9f6 100755 --- a/src/pm/hydra/mpichprereq +++ b/src/pm/hydra/mpichprereq @@ -10,7 +10,7 @@ if test -z "$PM_REQUIRES_PMI" ; then else PM_REQUIRES_PMI=simple fi -elif test "$PM_REQUIRES_PMI" != "simple" -a "$PM_REQUIRES_PMI" != "pmi2/simple" -a "$PM_REQUIRES_PMI" != "pmi2/poe"; then +elif test "$PM_REQUIRES_PMI" != "simple" -a "$PM_REQUIRES_PMI" != "pmi2/simple" ; then echo "hydra requires the \"simple\" or \"pmi2\" PMI implementation; \"$PM_REQUIRES_PMI\" has already been selected" exit 1 fi diff --git a/src/pmi/pmi2/Makefile.mk b/src/pmi/pmi2/Makefile.mk index fd3d7f4..460f67e 100644 --- a/src/pmi/pmi2/Makefile.mk +++ b/src/pmi/pmi2/Makefile.mk @@ -5,5 +5,4 @@ ## See COPYRIGHT in top-level directory. ## -include $(top_srcdir)/src/pmi/pmi2/poe/Makefile.mk include $(top_srcdir)/src/pmi/pmi2/simple/Makefile.mk diff --git a/src/pmi/pmi2/poe/Makefile.mk b/src/pmi/pmi2/poe/Makefile.mk deleted file mode 100644 index 26707a9..0000000 --- a/src/pmi/pmi2/poe/Makefile.mk +++ /dev/null @@ -1,15 +0,0 @@ -## -*- Mode: Makefile; -*- -## vim: set ft=automake : -## -## (C) 2011 by Argonne National Laboratory. -## See COPYRIGHT in top-level directory. -## - -if BUILD_PMI_PMI2_POE - -mpi_core_sources += \ - src/pmi/pmi2/poe/poe2pmi.c - -AM_CPPFLAGS += -I$(top_srcdir)/src/pmi/pmi2/poe - -endif BUILD_PMI_PMI2_POE diff --git a/src/pmi/pmi2/poe/poe2pmi.c b/src/pmi/pmi2/poe/poe2pmi.c deleted file mode 100644 index 28cc16f..0000000 --- a/src/pmi/pmi2/poe/poe2pmi.c +++ /dev/null @@ -1,449 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; -*- */ -/* - * (C) 2007 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - */ -#include <dlfcn.h> -#include "mpichconf.h" -#include "pmi2.h" -#include "mpiimpl.h" - -#include <stdio.h> -#ifdef HAVE_UNISTD_H -#include <unistd.h> -#endif -#ifdef HAVE_STDLIB_H -#include <stdlib.h> -#endif -#ifdef HAVE_STRING_H -#include <string.h> -#endif -#ifdef HAVE_STRINGS_H -#include <strings.h> -#endif -#if defined(HAVE_SYS_SOCKET_H) -#include <sys/socket.h> -#endif - -#ifdef USE_PMI_PORT -#ifndef MAXHOSTNAME -#define MAXHOSTNAME 256 -#endif -#endif - -#define PMII_EXIT_CODE -1 - -#define PMI_VERSION 2 -#define PMI_SUBVERSION 0 - -#define MAX_INT_STR_LEN 11 /* number of digits in MAX_UINT + 1 */ - -struct worldExitReq { - pami_work_t work; - int world_id; -}; - -int (*mp_world_exiting_handler)(int) = NULL; - -typedef enum { PMI2_UNINITIALIZED = 0, NORMAL_INIT_WITH_PM = 1 } PMI2State; -static PMI2State PMI2_initialized = PMI2_UNINITIALIZED; - -static int PMI2_debug = 0; -static int PMI2_fd = -1; -static int PMI2_size = 1; -static int PMI2_rank = 0; - -static int PMI2_debug_init = 0; /* Set this to true to debug the init */ - -int PMI2_pmiverbose = 0; /* Set this to true to print PMI debugging info */ - -extern MPIDI_PG_t *pg_world; -extern int world_rank; -extern int world_size; - -#ifdef MPICH_IS_THREADED -static MPID_Thread_mutex_t mutex; -static int blocked = FALSE; -static MPID_Thread_cond_t cond; -#endif - -extern int mpidi_finalized; -int _mpi_world_exiting_handler(int); - -void *poeptr = NULL; - -/* ------------------------------------------------------------------------- */ -/* PMI API Routines */ -/* ------------------------------------------------------------------------- */ -int PMI2_Init(int *spawned, int *size, int *rank, int *appnum) -{ - int pmi2_errno = PMI2_SUCCESS; - char *p; - char *jobid; - char *pmiid; - int ret; - - int (*pmi2_init)(int*, int*, int *, int*); - char *poelibname; - int poeflags; -#ifndef __AIX__ - poelibname = "libpoe.so"; - poeflags = RTLD_NOW|RTLD_GLOBAL; -#else - poeflags = RTLD_NOW|RTLD_GLOBAL|RTLD_MEMBER; - if(sizeof(void *) == 8) - poelibname = "libpoe_r.a(poe64_r.o)"; - else - poelibname = "libpoe_r.a(poe_r.o)"; -#endif - poeptr = dlopen(poelibname, poeflags); - if (poeptr == NULL) { - TRACE_ERR("failed to open %s error=%s\n", poelibname, dlerror()); - } - - pmi2_init = (int (*)())dlsym(poeptr, "PMI2_Init"); - if (pmi2_init == NULL) { - TRACE_ERR("failed to dlsym PMI2_Init\n"); - } - - ret = (*pmi2_init)(spawned, size, rank, appnum); - /*mp_world_exiting_handler = &(_mpi_world_exiting_handler); */ - return ret; -} - -int PMI2_Finalize(void) -{ - int pmi2_errno = PMI2_SUCCESS; - int rc; - const char *errmsg; - - int (*pmi2_finalize)(void); - - pmi2_finalize = (int (*)())dlsym(poeptr, "PMI2_Finalize"); - if (pmi2_finalize == NULL) { - TRACE_ERR("failed to dlsym PMI2_Finalize\n"); - } - - return (*pmi2_finalize)(); - -} - -int PMI2_Initialized(void) -{ - /* Turn this into a logical value (1 or 0) . This allows us - to use PMI2_initialized to distinguish between initialized with - an PMI service (e.g., via mpiexec) and the singleton init, - which has no PMI service */ - return PMI2_initialized != 0; -} - -int PMI2_Abort( int flag, const char msg[] ) -{ - int (*pmi2_abort)(int, const char*); - - pmi2_abort = (int (*)())dlsym(poeptr, "PMI2_Abort"); - if (pmi2_abort == NULL) { - TRACE_ERR("failed to dlsym pmi2_abort\n"); - } - - return (*pmi2_abort)(flag, msg); -} - -int PMI2_Job_Spawn(int count, const char * cmds[], - int argcs[], const char ** argvs[], - const int maxprocs[], - const int info_keyval_sizes[], - const struct MPIR_Info *info_keyval_vectors[], - int preput_keyval_size, - const struct MPIR_Info *preput_keyval_vector[], - char jobId[], int jobIdSize, - int errors[]) -{ - int i,rc,spawncnt,total_num_processes,num_errcodes_found; - int found; - const char *jid; - int jidlen; - char *lead, *lag; - int spawn_rc; - const char *errmsg = NULL; - int pmi2_errno = 0; - - int (*pmi2_job_spawn)(int , const char * [], int [], const char ** [],const int [],const int [],const struct MPIR_Info *[],int ,const struct MPIR_Info *[],char jobId[],int ,int []); - - pmi2_job_spawn = (int (*)())dlsym(poeptr, "PMI2_Job_Spawn"); - if (pmi2_job_spawn == NULL) { - TRACE_ERR("failed to dlsym pmi2_job_spawn\n"); - } - - return (*pmi2_job_spawn)(count, cmds, argcs, argvs, maxprocs, - info_keyval_sizes, info_keyval_vectors, - preput_keyval_size, preput_keyval_vector, - jobId, jobIdSize, errors); - -} - -int PMI2_Job_GetId(char jobid[], int jobid_size) -{ - int pmi2_errno = PMI2_SUCCESS; - int found; - const char *jid; - int jidlen; - int rc; - const char *errmsg; - - int (*pmi2_job_getid)(char*, int); - - pmi2_job_getid = (int (*)())dlsym(poeptr, "PMI2_Job_GetId"); - if (pmi2_job_getid == NULL) { - TRACE_ERR("failed to dlsym pmi2_job_getid\n"); - } - - return (*pmi2_job_getid)(jobid, jobid_size); -} - - -int PMI2_KVS_Put(const char key[], const char value[]) -{ - int pmi2_errno = PMI2_SUCCESS; - int rc; - const char *errmsg; - - int (*pmi2_kvs_put)(const char*, const char*); - - pmi2_kvs_put = (int (*)())dlsym(poeptr, "PMI2_KVS_Put"); - if (pmi2_kvs_put == NULL) { - TRACE_ERR("failed to dlsym pmi2_kvs_put\n"); - } - - return (*pmi2_kvs_put)(key, value); -} - -int PMI2_KVS_Fence(void) -{ - int pmi2_errno = PMI2_SUCCESS; - int rc; - const char *errmsg; - - int (*pmi2_kvs_fence)(void); - - pmi2_kvs_fence = (int (*)())dlsym(poeptr, "PMI2_KVS_Fence"); - if (pmi2_kvs_fence == NULL) { - TRACE_ERR("failed to dlsym pmi2_kvs_fence\n"); - } - - return (*pmi2_kvs_fence)(); -} - -int PMI2_KVS_Get(const char *jobid, int src_pmi_id, const char key[], char value [], int maxValue, int *valLen) -{ - int pmi2_errno = PMI2_SUCCESS; - int found, keyfound; - const char *kvsvalue; - int kvsvallen; - int ret; - int rc; - char src_pmi_id_str[256]; - const char *errmsg; - - int (*pmi2_kvs_get)(const char*, int, const char *, char *, int, int*); - - pmi2_kvs_get = (int (*)())dlsym(poeptr, "PMI2_KVS_Get"); - if (pmi2_kvs_get == NULL) { - TRACE_ERR("failed to dlsym pmi2_kvs_get\n"); - } - - return (*pmi2_kvs_get)(jobid, src_pmi_id, key, value, maxValue, valLen); -} - - -int PMI2_Info_GetJobAttr(const char name[], char value[], int valuelen, int *flag) -{ - int pmi2_errno = PMI2_SUCCESS; - int found; - const char *kvsvalue; - int kvsvallen; - int rc; - const char *errmsg; - - int (*pmi2_info_getjobattr)(const char*, char *, int, int*); - - pmi2_info_getjobattr = (int (*)())dlsym(poeptr, "PMI2_Info_GetJobAttr"); - if (pmi2_info_getjobattr == NULL) { - TRACE_ERR("failed to dlsym pmi2_info_getjobattr\n"); - } - - return (*pmi2_info_getjobattr)(name, value, valuelen, flag); -} - - -/** - * This is the mpi level of callback that get invoked when a task get notified - * of a world's exiting - */ -int _mpi_world_exiting_handler(int world_id) -{ - /* check the reference count associated with that remote world - if the reference count is zero, the task will call LAPI_Purge_totask on - all tasks in that world,reset MPCI. It would also remove the world - structure corresponding to that world ID - if the reference count is not zero, it should call STOPALL - */ - int rc,ref_count = -1; - int *taskid_list = NULL; - int i; - int my_state=FALSE,reduce_state=FALSE; - char world_id_str[32]; - int mpi_errno = MPI_SUCCESS; - pami_endpoint_t dest; -/* struct worldExitReq *req = (struct worldExitReq *)cookie; */ - MPIR_Comm *comm = MPIR_Process.comm_world; - - MPID_THREAD_CS_ENTER(GLOBAL, MPIR_THREAD_GLOBAL_ALLFUNC_MUTEX); - ref_count = MPIDI_get_refcnt_of_world(world_id); - TRACE_ERR("_mpi_world_exiting_handler: invoked for world %d exiting ref_count=%d my comm_word_size=%d\n", world_id, ref_count, world_size); - if(ref_count == 0) { - taskid_list = MPIDI_get_taskids_in_world_id(world_id); - if(taskid_list != NULL) { - for(i=0;taskid_list[i]!=-1;i++) { - PAMI_Endpoint_create(MPIDI_Client, taskid_list[i], 0, &dest); - MPIDI_OpState_reset(taskid_list[i]); - MPIDI_IpState_reset(taskid_list[i]); - TRACE_ERR("PAMI_Purge on taskid_list[%d]=%d\n", i,taskid_list[i]); - PAMI_Purge(MPIDI_Context[0], &dest, 1); - } - MPIDI_delete_conn_record(world_id); - } - rc = -1; - } - my_state = TRUE; - - rc = _mpi_reduce_for_dyntask(&my_state, &reduce_state); - if(rc) return rc; - - TRACE_ERR("_mpi_world_exiting_handler: Out of _mpi_reduce_for_dyntask for exiting world %d reduce_state=%d\n",world_id, reduce_state); - - MPID_THREAD_CS_EXIT(GLOBAL, MPIR_THREAD_GLOBAL_ALLFUNC_MUTEX); - if(comm->rank == 0) { - MPL_snprintf(world_id_str, sizeof(world_id_str), "%d", world_id); - PMI2_Abort(0, world_id_str); - if((reduce_state != world_size)) { - TRACE_ERR("root is exiting with error\n"); - exit(-1); - } - TRACE_ERR("_mpi_world_exiting_handler: Root finished sending SSM_WORLD_EXITING to POE for exiting world %d\n",world_id); - } - - if(ref_count != 0) { - TRACE_ERR("STOPALL is sent by task %d\n", PAMIX_Client_query(MPIDI_Client, PAMI_CLIENT_TASK_ID ).value.intval); - PMI2_Abort(1, "STOPALL should be sent"); - rc = -2; - } - -/* if(cookie) MPL_free(cookie);*/ - return PAMI_SUCCESS; -} - - -int getchildren(int iam, double alpha,int gsize, int *children, - int *blocks, int *numchildren, int *parent) -{ - int fakeme=iam,i; - int p=gsize,pbig,bflag=0,blocks_from_children=0; - - *numchildren=0; - - if( blocks != NULL ) - bflag=1; - - while( p > 1 ) { - - pbig = MPL_MAX(1,MPL_MIN((int) (alpha*(double)p), p-1)); - - if ( fakeme == 0 ) { - - (children)[*numchildren] = (iam+pbig+gsize)%gsize; - if(bflag) - (blocks)[*numchildren] = p -pbig; - - *numchildren +=1; - } - if ( fakeme == pbig ) { - *parent = (iam-pbig+gsize)%gsize; - if(bflag) - blocks_from_children = p - pbig; - } - if( pbig > fakeme) { - p = pbig; - } else { - p -=pbig; - fakeme -=pbig; - } - } - if(bflag) - (blocks)[*numchildren] = blocks_from_children; -} - -int _mpi_reduce_for_dyntask(int *sendbuf, int *recvbuf) -{ - int *children, gid, child_rank, parent_rank, rc; - int numchildren, parent=0, i, result=0,tag, remaining_child_count; - MPIR_Comm *comm_ptr; - int mpi_errno; - MPIR_Errflag_t errflag = MPIR_ERR_NONE; - - int TASKS= world_size; - children = MPL_malloc(TASKS*sizeof(int)); - - comm_ptr = MPIR_Process.comm_world; - - if(pg_world && pg_world->id) - tag = (-1) * (atoi(pg_world->id)); - else { - TRACE_ERR("pg_world hasn't been created, should skip the rest of the handler and return\n"); - return -1; - } - - result = *sendbuf; - - getchildren(world_rank, 0.5, TASKS, children, NULL, &numchildren, &parent); - - TRACE_ERR("_mpi_reduce_for_dyntask - numchildren=%d parent=%d world_rank=%d\n", numchildren, parent, world_rank); - for(i=numchildren-1;i>=0;i--) - { - remaining_child_count = i; - child_rank = (children[i])% TASKS; - TRACE_ERR("_mpi_reduce_for_dyntask - recv from child_rank%d child_taskid=%d\n", child_rank, pg_world->vct[child_rank].taskid); - mpi_errno = MPIC_Recv(recvbuf, sizeof(int),MPI_BYTE, child_rank, tag, comm_ptr, MPI_STATUS_IGNORE, &errflag); - TRACE_ERR("_mpi_reduce_for_dyntask - recv DONE from child_rank%d child_taskid=%d\n", child_rank, pg_world->vct[child_rank].taskid); - - if(world_rank != parent) - { - if(remaining_child_count == 0) { - parent_rank = (parent) % TASKS; - result += *recvbuf; - TRACE_ERR("_mpi_reduce_for_dyntask - send to parent_rank=%d parent taskid=%d \n", parent_rank, pg_world->vct[parent_rank].taskid); - MPIC_Send(&result, sizeof(int), MPI_BYTE, parent_rank, tag, comm_ptr, &errflag); - } - else - { - result += *recvbuf; - } - } - if(world_rank == 0) - { - result += *recvbuf; - } - } - - if(world_rank != parent && numchildren == 0) { - parent_rank = (parent) % TASKS; - TRACE_ERR("_mpi_reduce_for_dyntask - send to parent_rank=%d parent_task_id=%d\n", parent_rank, pg_world->vct[parent_rank].taskid); - MPIC_Send(sendbuf, sizeof(int), MPI_BYTE, parent_rank, tag, comm_ptr, &errflag); - } - - if(world_rank == 0) { - *recvbuf = result; - } - MPL_free(children); - return 0; -} diff --git a/src/pmi/pmi2/poe/subconfigure.m4 b/src/pmi/pmi2/poe/subconfigure.m4 deleted file mode 100644 index 5e2ce7d..0000000 --- a/src/pmi/pmi2/poe/subconfigure.m4 +++ /dev/null @@ -1,24 +0,0 @@ -[#] start of __file__ -dnl MPICH_SUBCFG_AFTER=src/pmi - -AC_DEFUN([PAC_SUBCFG_PREREQ_]PAC_SUBCFG_AUTO_SUFFIX,[ -]) - -AC_DEFUN([PAC_SUBCFG_BODY_]PAC_SUBCFG_AUTO_SUFFIX,[ - -AM_CONDITIONAL([BUILD_PMI_PMI2_POE],[test "x$pmi_name" = "xpmi2/poe"]) - -AM_COND_IF([BUILD_PMI_PMI2_POE],[ -if test "$enable_pmiport" != "no" ; then - enable_pmiport=yes -fi - -dnl causes USE_PMI2_API to be AC_DEFINE'ed by the top-level configure.ac -USE_PMI2_API=yes - -PAC_C_GNU_ATTRIBUTE -])dnl end COND_IF - -])dnl end BODY macro - -[#] end of __file__ diff --git a/src/pmi/pmi2/subconfigure.m4 b/src/pmi/pmi2/subconfigure.m4 index d1dc441..c516051 100644 --- a/src/pmi/pmi2/subconfigure.m4 +++ b/src/pmi/pmi2/subconfigure.m4 @@ -16,7 +16,6 @@ fi dnl causes USE_PMI2_API to be AC_DEFINE'ed by the top-level configure.ac USE_PMI2_API=yes -# common ARG_ENABLE, shared by "simple" and "poe" AC_ARG_ENABLE(pmiport, [--enable-pmiport - Allow PMI interface to use a host-port pair to contact for PMI services],,enable_pmiport=default) ----------------------------------------------------------------------- Summary of changes: CHANGES | 22 + README.vin | 27 - maint/version.m4 | 4 +- src/mpid/Makefile.mk | 1 - src/mpid/pamid/.gitattributes | 15 - src/mpid/pamid/.gitignore | 3 - src/mpid/pamid/Makefile.mk | 43 - src/mpid/pamid/cross/bgq8 | 27 - src/mpid/pamid/cross/pe4 | 27 - src/mpid/pamid/cross/pe4-aix | 28 - src/mpid/pamid/cross/pe8 | 27 - src/mpid/pamid/cross/pe8-aix | 28 - src/mpid/pamid/include/.gitignore | 1 - src/mpid/pamid/include/mpidi_constants.h | 128 -- src/mpid/pamid/include/mpidi_datatypes.h | 658 ------- src/mpid/pamid/include/mpidi_externs.h | 43 - src/mpid/pamid/include/mpidi_hooks.h | 66 - src/mpid/pamid/include/mpidi_macros.h | 217 -- src/mpid/pamid/include/mpidi_mutex.h | 374 ---- src/mpid/pamid/include/mpidi_platform.h | 196 -- src/mpid/pamid/include/mpidi_prototypes.h | 460 ----- src/mpid/pamid/include/mpidi_thread.h | 184 -- src/mpid/pamid/include/mpidi_trace.h | 266 --- src/mpid/pamid/include/mpidi_util.h | 187 -- src/mpid/pamid/include/mpidimpl.h | 197 -- src/mpid/pamid/include/mpidpost.h | 104 - src/mpid/pamid/include/mpidpre.h | 94 - src/mpid/pamid/include/mpix.h | 298 --- src/mpid/pamid/include/pamix.h | 147 -- src/mpid/pamid/src/Makefile.mk | 73 - src/mpid/pamid/src/coll/Makefile.mk | 47 - src/mpid/pamid/src/coll/allgather/Makefile.mk | 30 - .../pamid/src/coll/allgather/mpido_allgather.c | 820 -------- .../pamid/src/coll/allgather/mpido_iallgather.c | 73 - src/mpid/pamid/src/coll/allgatherv/Makefile.mk | 30 - .../pamid/src/coll/allgatherv/mpido_allgatherv.c | 921 --------- .../pamid/src/coll/allgatherv/mpido_iallgatherv.c | 77 - src/mpid/pamid/src/coll/allreduce/Makefile.mk | 30 - .../pamid/src/coll/allreduce/mpido_allreduce.c | 562 ------ .../pamid/src/coll/allreduce/mpido_iallreduce.c | 73 - src/mpid/pamid/src/coll/alltoall/Makefile.mk | 30 - src/mpid/pamid/src/coll/alltoall/mpido_alltoall.c | 449 ----- src/mpid/pamid/src/coll/alltoall/mpido_ialltoall.c | 76 - src/mpid/pamid/src/coll/alltoallv/Makefile.mk | 30 - .../pamid/src/coll/alltoallv/mpido_alltoallv.c | 734 ------- .../pamid/src/coll/alltoallv/mpido_ialltoallv.c | 79 - src/mpid/pamid/src/coll/alltoallw/Makefile.mk | 28 - .../pamid/src/coll/alltoallw/mpido_ialltoallw.c | 79 - src/mpid/pamid/src/coll/barrier/Makefile.mk | 30 - src/mpid/pamid/src/coll/barrier/mpido_barrier.c | 146 -- src/mpid/pamid/src/coll/barrier/mpido_ibarrier.c | 62 - src/mpid/pamid/src/coll/bcast/Makefile.mk | 30 - src/mpid/pamid/src/coll/bcast/mpido_bcast.c | 394 ---- src/mpid/pamid/src/coll/bcast/mpido_ibcast.c | 67 - src/mpid/pamid/src/coll/coll_utils.c | 405 ---- src/mpid/pamid/src/coll/exscan/Makefile.mk | 28 - src/mpid/pamid/src/coll/exscan/mpido_iexscan.c | 65 - src/mpid/pamid/src/coll/gather/Makefile.mk | 30 - src/mpid/pamid/src/coll/gather/mpido_gather.c | 642 ------ src/mpid/pamid/src/coll/gather/mpido_igather.c | 72 - src/mpid/pamid/src/coll/gatherv/Makefile.mk | 30 - src/mpid/pamid/src/coll/gatherv/mpido_gatherv.c | 573 ------ src/mpid/pamid/src/coll/gatherv/mpido_igatherv.c | 78 - src/mpid/pamid/src/coll/ired_scat/Makefile.mk | 28 - .../pamid/src/coll/ired_scat/mpido_ired_scat.c | 72 - .../pamid/src/coll/ired_scat_block/Makefile.mk | 28 - .../coll/ired_scat_block/mpido_ired_scat_block.c | 72 - src/mpid/pamid/src/coll/red_scat/Makefile.mk | 29 - src/mpid/pamid/src/coll/red_scat/mpido_red_scat.c | 185 -- src/mpid/pamid/src/coll/reduce/Makefile.mk | 30 - src/mpid/pamid/src/coll/reduce/mpido_ireduce.c | 72 - src/mpid/pamid/src/coll/reduce/mpido_reduce.c | 422 ---- src/mpid/pamid/src/coll/scan/Makefile.mk | 29 - src/mpid/pamid/src/coll/scan/mpido_scan.c | 452 ----- src/mpid/pamid/src/coll/scatter/Makefile.mk | 30 - src/mpid/pamid/src/coll/scatter/mpido_iscatter.c | 74 - src/mpid/pamid/src/coll/scatter/mpido_scatter.c | 591 ------ src/mpid/pamid/src/coll/scatterv/Makefile.mk | 30 - src/mpid/pamid/src/coll/scatterv/mpido_iscatterv.c | 73 - src/mpid/pamid/src/coll/scatterv/mpido_scatterv.c | 812 -------- src/mpid/pamid/src/comm/Makefile.mk | 31 - src/mpid/pamid/src/comm/mpid_comm.c | 432 ---- src/mpid/pamid/src/comm/mpid_optcolls.c | 1047 ---------- src/mpid/pamid/src/comm/mpid_selectcolls.c | 746 ------- src/mpid/pamid/src/dyntask/Makefile.mk | 32 - src/mpid/pamid/src/dyntask/mpid_comm_disconnect.c | 541 ----- .../pamid/src/dyntask/mpid_comm_spawn_multiple.c | 414 ---- src/mpid/pamid/src/dyntask/mpid_port.c | 317 --- src/mpid/pamid/src/dyntask/mpidi_pg.c | 1063 ---------- src/mpid/pamid/src/dyntask/mpidi_port.c | 1766 ----------------- src/mpid/pamid/src/misc/Makefile.mk | 33 - src/mpid/pamid/src/misc/mpid_abort.c | 103 - src/mpid/pamid/src/misc/mpid_get_universe_size.c | 96 - src/mpid/pamid/src/misc/mpid_getpname.c | 46 - src/mpid/pamid/src/misc/mpid_mem.c | 42 - src/mpid/pamid/src/misc/mpid_unimpl.c | 112 -- src/mpid/pamid/src/mpid_aint.c | 61 - src/mpid/pamid/src/mpid_buffer.c | 250 --- src/mpid/pamid/src/mpid_finalize.c | 149 -- src/mpid/pamid/src/mpid_improbe.c | 49 - src/mpid/pamid/src/mpid_imrecv.c | 91 - src/mpid/pamid/src/mpid_init.c | 1703 ---------------- src/mpid/pamid/src/mpid_iprobe.c | 47 - src/mpid/pamid/src/mpid_mprobe.c | 41 - src/mpid/pamid/src/mpid_mrecv.c | 34 - src/mpid/pamid/src/mpid_probe.c | 41 - src/mpid/pamid/src/mpid_progress.c | 355 ---- src/mpid/pamid/src/mpid_progress.h | 297 --- src/mpid/pamid/src/mpid_recvq.c | 945 --------- src/mpid/pamid/src/mpid_recvq.h | 425 ---- src/mpid/pamid/src/mpid_recvq_mmap.cpp | 419 ---- src/mpid/pamid/src/mpid_request.c | 88 - src/mpid/pamid/src/mpid_request.h | 354 ---- src/mpid/pamid/src/mpid_time.c | 112 -- src/mpid/pamid/src/mpid_vc.c | 360 ---- src/mpid/pamid/src/mpidi_bufmm.c | 716 ------- src/mpid/pamid/src/mpidi_env.c | 1314 ------------- src/mpid/pamid/src/mpidi_mutex.c | 88 - src/mpid/pamid/src/mpidi_nbc_sched.c | 65 - src/mpid/pamid/src/mpidi_pami_datatype.c | 155 -- src/mpid/pamid/src/mpidi_util.c | 2057 -------------------- src/mpid/pamid/src/mpix/Makefile.mk | 29 - src/mpid/pamid/src/mpix/mpix.c | 1210 ------------ src/mpid/pamid/src/onesided/Makefile.mk | 57 - src/mpid/pamid/src/onesided/mpid_1s.c | 151 -- src/mpid/pamid/src/onesided/mpid_win_accumulate.c | 364 ---- src/mpid/pamid/src/onesided/mpid_win_allocate.c | 99 - .../pamid/src/onesided/mpid_win_allocate_shared.c | 561 ------ src/mpid/pamid/src/onesided/mpid_win_attach.c | 58 - .../pamid/src/onesided/mpid_win_compare_and_swap.c | 164 -- src/mpid/pamid/src/onesided/mpid_win_create.c | 213 -- .../pamid/src/onesided/mpid_win_create_dynamic.c | 90 - src/mpid/pamid/src/onesided/mpid_win_detach.c | 53 - src/mpid/pamid/src/onesided/mpid_win_fence.c | 71 - .../pamid/src/onesided/mpid_win_fetch_and_op.c | 378 ---- src/mpid/pamid/src/onesided/mpid_win_flush.c | 180 -- src/mpid/pamid/src/onesided/mpid_win_free.c | 132 -- src/mpid/pamid/src/onesided/mpid_win_get.c | 434 ----- .../pamid/src/onesided/mpid_win_get_accumulate.c | 586 ------ src/mpid/pamid/src/onesided/mpid_win_get_info.c | 104 - src/mpid/pamid/src/onesided/mpid_win_lock.c | 230 --- src/mpid/pamid/src/onesided/mpid_win_lock_all.c | 206 -- src/mpid/pamid/src/onesided/mpid_win_pscw.c | 270 --- src/mpid/pamid/src/onesided/mpid_win_put.c | 434 ----- src/mpid/pamid/src/onesided/mpid_win_reqops.c | 253 --- src/mpid/pamid/src/onesided/mpid_win_set_info.c | 102 - .../pamid/src/onesided/mpid_win_shared_query.c | 65 - src/mpid/pamid/src/onesided/mpid_win_sync.c | 54 - src/mpid/pamid/src/onesided/mpidi_onesided.h | 389 ---- src/mpid/pamid/src/onesided/mpidi_win_control.c | 149 -- src/mpid/pamid/src/pamix/Makefile.mk | 29 - src/mpid/pamid/src/pamix/pamix.c | 347 ---- src/mpid/pamid/src/pt2pt/Makefile.mk | 50 - src/mpid/pamid/src/pt2pt/mpid_cancel.c | 138 -- src/mpid/pamid/src/pt2pt/mpid_irecv.h | 67 - src/mpid/pamid/src/pt2pt/mpid_isend.h | 129 -- src/mpid/pamid/src/pt2pt/mpid_issend.c | 62 - src/mpid/pamid/src/pt2pt/mpid_recv.c | 60 - src/mpid/pamid/src/pt2pt/mpid_send.h | 67 - src/mpid/pamid/src/pt2pt/mpid_ssend.c | 58 - src/mpid/pamid/src/pt2pt/mpidi_callback_eager.c | 312 --- src/mpid/pamid/src/pt2pt/mpidi_callback_rzv.c | 234 --- src/mpid/pamid/src/pt2pt/mpidi_callback_short.c | 268 --- src/mpid/pamid/src/pt2pt/mpidi_callback_util.c | 195 -- src/mpid/pamid/src/pt2pt/mpidi_control.c | 414 ---- src/mpid/pamid/src/pt2pt/mpidi_done.c | 316 --- src/mpid/pamid/src/pt2pt/mpidi_recv.h | 257 --- src/mpid/pamid/src/pt2pt/mpidi_recvmsg.c | 161 -- src/mpid/pamid/src/pt2pt/mpidi_rendezvous.c | 266 --- src/mpid/pamid/src/pt2pt/mpidi_send.h | 128 -- src/mpid/pamid/src/pt2pt/mpidi_sendmsg.c | 743 ------- src/mpid/pamid/src/pt2pt/persistent/Makefile.mk | 30 - .../pamid/src/pt2pt/persistent/mpid_recv_init.c | 56 - .../pamid/src/pt2pt/persistent/mpid_send_init.c | 148 -- .../pamid/src/pt2pt/persistent/mpid_startall.c | 140 -- src/mpid/pamid/subconfigure.m4 | 351 ---- src/mpl/configure.ac | 1 - src/pm/hydra/mpichprereq | 2 +- src/pmi/pmi2/Makefile.mk | 1 - src/pmi/pmi2/poe/Makefile.mk | 15 - src/pmi/pmi2/poe/poe2pmi.c | 449 ----- src/pmi/pmi2/poe/subconfigure.m4 | 24 - src/pmi/pmi2/subconfigure.m4 | 1 - 183 files changed, 25 insertions(+), 43411 deletions(-) delete mode 100644 src/mpid/pamid/.gitattributes delete mode 100644 src/mpid/pamid/.gitignore delete mode 100644 src/mpid/pamid/Makefile.mk delete mode 100644 src/mpid/pamid/cross/bgq8 delete mode 100644 src/mpid/pamid/cross/pe4 delete mode 100644 src/mpid/pamid/cross/pe4-aix delete mode 100644 src/mpid/pamid/cross/pe8 delete mode 100644 src/mpid/pamid/cross/pe8-aix delete mode 100644 src/mpid/pamid/include/.gitignore delete mode 100644 src/mpid/pamid/include/mpidi_constants.h delete mode 100644 src/mpid/pamid/include/mpidi_datatypes.h delete mode 100644 src/mpid/pamid/include/mpidi_externs.h delete mode 100644 src/mpid/pamid/include/mpidi_hooks.h delete mode 100644 src/mpid/pamid/include/mpidi_macros.h delete mode 100644 src/mpid/pamid/include/mpidi_mutex.h delete mode 100644 src/mpid/pamid/include/mpidi_platform.h delete mode 100644 src/mpid/pamid/include/mpidi_prototypes.h delete mode 100644 src/mpid/pamid/include/mpidi_thread.h delete mode 100644 src/mpid/pamid/include/mpidi_trace.h delete mode 100644 src/mpid/pamid/include/mpidi_util.h delete mode 100644 src/mpid/pamid/include/mpidimpl.h delete mode 100644 src/mpid/pamid/include/mpidpost.h delete mode 100644 src/mpid/pamid/include/mpidpre.h delete mode 100644 src/mpid/pamid/include/mpix.h delete mode 100644 src/mpid/pamid/include/pamix.h delete mode 100644 src/mpid/pamid/src/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/allgather/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/allgather/mpido_allgather.c delete mode 100644 src/mpid/pamid/src/coll/allgather/mpido_iallgather.c delete mode 100644 src/mpid/pamid/src/coll/allgatherv/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/allgatherv/mpido_allgatherv.c delete mode 100644 src/mpid/pamid/src/coll/allgatherv/mpido_iallgatherv.c delete mode 100644 src/mpid/pamid/src/coll/allreduce/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/allreduce/mpido_allreduce.c delete mode 100644 src/mpid/pamid/src/coll/allreduce/mpido_iallreduce.c delete mode 100644 src/mpid/pamid/src/coll/alltoall/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/alltoall/mpido_alltoall.c delete mode 100644 src/mpid/pamid/src/coll/alltoall/mpido_ialltoall.c delete mode 100644 src/mpid/pamid/src/coll/alltoallv/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/alltoallv/mpido_alltoallv.c delete mode 100644 src/mpid/pamid/src/coll/alltoallv/mpido_ialltoallv.c delete mode 100644 src/mpid/pamid/src/coll/alltoallw/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/alltoallw/mpido_ialltoallw.c delete mode 100644 src/mpid/pamid/src/coll/barrier/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/barrier/mpido_barrier.c delete mode 100644 src/mpid/pamid/src/coll/barrier/mpido_ibarrier.c delete mode 100644 src/mpid/pamid/src/coll/bcast/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/bcast/mpido_bcast.c delete mode 100644 src/mpid/pamid/src/coll/bcast/mpido_ibcast.c delete mode 100644 src/mpid/pamid/src/coll/coll_utils.c delete mode 100644 src/mpid/pamid/src/coll/exscan/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/exscan/mpido_iexscan.c delete mode 100644 src/mpid/pamid/src/coll/gather/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/gather/mpido_gather.c delete mode 100644 src/mpid/pamid/src/coll/gather/mpido_igather.c delete mode 100644 src/mpid/pamid/src/coll/gatherv/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/gatherv/mpido_gatherv.c delete mode 100644 src/mpid/pamid/src/coll/gatherv/mpido_igatherv.c delete mode 100644 src/mpid/pamid/src/coll/ired_scat/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/ired_scat/mpido_ired_scat.c delete mode 100644 src/mpid/pamid/src/coll/ired_scat_block/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/ired_scat_block/mpido_ired_scat_block.c delete mode 100644 src/mpid/pamid/src/coll/red_scat/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/red_scat/mpido_red_scat.c delete mode 100644 src/mpid/pamid/src/coll/reduce/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/reduce/mpido_ireduce.c delete mode 100644 src/mpid/pamid/src/coll/reduce/mpido_reduce.c delete mode 100644 src/mpid/pamid/src/coll/scan/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/scan/mpido_scan.c delete mode 100644 src/mpid/pamid/src/coll/scatter/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/scatter/mpido_iscatter.c delete mode 100644 src/mpid/pamid/src/coll/scatter/mpido_scatter.c delete mode 100644 src/mpid/pamid/src/coll/scatterv/Makefile.mk delete mode 100644 src/mpid/pamid/src/coll/scatterv/mpido_iscatterv.c delete mode 100644 src/mpid/pamid/src/coll/scatterv/mpido_scatterv.c delete mode 100644 src/mpid/pamid/src/comm/Makefile.mk delete mode 100644 src/mpid/pamid/src/comm/mpid_comm.c delete mode 100644 src/mpid/pamid/src/comm/mpid_optcolls.c delete mode 100644 src/mpid/pamid/src/comm/mpid_selectcolls.c delete mode 100644 src/mpid/pamid/src/dyntask/Makefile.mk delete mode 100644 src/mpid/pamid/src/dyntask/mpid_comm_disconnect.c delete mode 100644 src/mpid/pamid/src/dyntask/mpid_comm_spawn_multiple.c delete mode 100644 src/mpid/pamid/src/dyntask/mpid_port.c delete mode 100644 src/mpid/pamid/src/dyntask/mpidi_pg.c delete mode 100644 src/mpid/pamid/src/dyntask/mpidi_port.c delete mode 100644 src/mpid/pamid/src/misc/Makefile.mk delete mode 100644 src/mpid/pamid/src/misc/mpid_abort.c delete mode 100644 src/mpid/pamid/src/misc/mpid_get_universe_size.c delete mode 100644 src/mpid/pamid/src/misc/mpid_getpname.c delete mode 100644 src/mpid/pamid/src/misc/mpid_mem.c delete mode 100644 src/mpid/pamid/src/misc/mpid_unimpl.c delete mode 100644 src/mpid/pamid/src/mpid_aint.c delete mode 100644 src/mpid/pamid/src/mpid_buffer.c delete mode 100644 src/mpid/pamid/src/mpid_finalize.c delete mode 100644 src/mpid/pamid/src/mpid_improbe.c delete mode 100644 src/mpid/pamid/src/mpid_imrecv.c delete mode 100644 src/mpid/pamid/src/mpid_init.c delete mode 100644 src/mpid/pamid/src/mpid_iprobe.c delete mode 100644 src/mpid/pamid/src/mpid_mprobe.c delete mode 100644 src/mpid/pamid/src/mpid_mrecv.c delete mode 100644 src/mpid/pamid/src/mpid_probe.c delete mode 100644 src/mpid/pamid/src/mpid_progress.c delete mode 100644 src/mpid/pamid/src/mpid_progress.h delete mode 100644 src/mpid/pamid/src/mpid_recvq.c delete mode 100644 src/mpid/pamid/src/mpid_recvq.h delete mode 100644 src/mpid/pamid/src/mpid_recvq_mmap.cpp delete mode 100644 src/mpid/pamid/src/mpid_request.c delete mode 100644 src/mpid/pamid/src/mpid_request.h delete mode 100644 src/mpid/pamid/src/mpid_time.c delete mode 100644 src/mpid/pamid/src/mpid_vc.c delete mode 100644 src/mpid/pamid/src/mpidi_bufmm.c delete mode 100644 src/mpid/pamid/src/mpidi_env.c delete mode 100644 src/mpid/pamid/src/mpidi_mutex.c delete mode 100644 src/mpid/pamid/src/mpidi_nbc_sched.c delete mode 100644 src/mpid/pamid/src/mpidi_pami_datatype.c delete mode 100644 src/mpid/pamid/src/mpidi_util.c delete mode 100644 src/mpid/pamid/src/mpix/Makefile.mk delete mode 100644 src/mpid/pamid/src/mpix/mpix.c delete mode 100644 src/mpid/pamid/src/onesided/Makefile.mk delete mode 100644 src/mpid/pamid/src/onesided/mpid_1s.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_accumulate.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_allocate.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_allocate_shared.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_attach.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_compare_and_swap.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_create.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_create_dynamic.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_detach.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_fence.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_fetch_and_op.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_flush.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_free.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_get.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_get_accumulate.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_get_info.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_lock.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_lock_all.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_pscw.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_put.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_reqops.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_set_info.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_shared_query.c delete mode 100644 src/mpid/pamid/src/onesided/mpid_win_sync.c delete mode 100644 src/mpid/pamid/src/onesided/mpidi_onesided.h delete mode 100644 src/mpid/pamid/src/onesided/mpidi_win_control.c delete mode 100644 src/mpid/pamid/src/pamix/Makefile.mk delete mode 100644 src/mpid/pamid/src/pamix/pamix.c delete mode 100644 src/mpid/pamid/src/pt2pt/Makefile.mk delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_cancel.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_irecv.h delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_isend.h delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_issend.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_recv.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_send.h delete mode 100644 src/mpid/pamid/src/pt2pt/mpid_ssend.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_callback_eager.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_callback_rzv.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_callback_short.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_callback_util.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_control.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_done.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_recv.h delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_recvmsg.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_rendezvous.c delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_send.h delete mode 100644 src/mpid/pamid/src/pt2pt/mpidi_sendmsg.c delete mode 100644 src/mpid/pamid/src/pt2pt/persistent/Makefile.mk delete mode 100644 src/mpid/pamid/src/pt2pt/persistent/mpid_recv_init.c delete mode 100644 src/mpid/pamid/src/pt2pt/persistent/mpid_send_init.c delete mode 100644 src/mpid/pamid/src/pt2pt/persistent/mpid_startall.c delete mode 100644 src/mpid/pamid/subconfigure.m4 delete mode 100644 src/pmi/pmi2/poe/Makefile.mk delete mode 100644 src/pmi/pmi2/poe/poe2pmi.c delete mode 100644 src/pmi/pmi2/poe/subconfigure.m4 hooks/post-receive -- MPICH primary repository