September 2015 - devel - mailman3.cels.anl.gov

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Rob Latham 04 Sep '15

04 Sep '15

On 09/04/2015 10:16 AM, Jeff Squyres (jsquyres) wrote: > Just curious, what's the process for getting this patch accepted into MPICH? > We have a formal workflow for frequent contributors that involves sign-offs and such. For occasional contributions, sending a patch to devel@mpich (as you did) works great. Ken's going to review this. ==rob > Thanks. > > >> On Sep 3, 2015, at 1:24 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >> >> Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, >> you really need to specify FI_ORDER_SAS for both transmit and receive >> if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; >> the MPI-style tagging interface doesn't make much sense unless the >> underlying fabric provider guarnatees ordering). >> >> Signed-off-by: Jeff Squyres <jsquyres(a)cisco.com> >> --- >> src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c | 10 ++++++---- >> 1 file changed, 6 insertions(+), 4 deletions(-) >> >> diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c >> index 5de47b7..2cf3337 100644 >> --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c >> +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c >> @@ -77,10 +77,12 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ >> /* We expect to register all memory up front for use with this */ >> /* endpoint, so the netmod requires dynamic memory regions */ >> /* ------------------------------------------------------------------------ */ >> - hints = fi_allocinfo(); >> - hints->mode = FI_CONTEXT; >> - hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ >> - hints->caps = FI_TAGGED; /* Tag matching interface */ >> + hints = fi_allocinfo(); >> + hints->mode = FI_CONTEXT; >> + hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ >> + hints->caps = FI_TAGGED; /* Tag matching interface */ >> + hints->tx_attr->msg_order = FI_ORDER_SAS; >> + hints->rx_attr->msg_order = FI_ORDER_SAS; >> >> hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; >> MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); >> -- >> 2.4.3 >> > > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres (jsquyres) 04 Sep '15

04 Sep '15

Just curious, what's the process for getting this patch accepted into MPICH? Thanks. > On Sep 3, 2015, at 1:24 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > > Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, > you really need to specify FI_ORDER_SAS for both transmit and receive > if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; > the MPI-style tagging interface doesn't make much sense unless the > underlying fabric provider guarnatees ordering). > > Signed-off-by: Jeff Squyres <jsquyres(a)cisco.com> > --- > src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c | 10 ++++++---- > 1 file changed, 6 insertions(+), 4 deletions(-) > > diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c > index 5de47b7..2cf3337 100644 > --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c > +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c > @@ -77,10 +77,12 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ > /* We expect to register all memory up front for use with this */ > /* endpoint, so the netmod requires dynamic memory regions */ > /* ------------------------------------------------------------------------ */ > - hints = fi_allocinfo(); > - hints->mode = FI_CONTEXT; > - hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ > - hints->caps = FI_TAGGED; /* Tag matching interface */ > + hints = fi_allocinfo(); > + hints->mode = FI_CONTEXT; > + hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ > + hints->caps = FI_TAGGED; /* Tag matching interface */ > + hints->tx_attr->msg_order = FI_ORDER_SAS; > + hints->rx_attr->msg_order = FI_ORDER_SAS; > > hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; > MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); > -- > 2.4.3 > -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

[PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres 03 Sep '15

03 Sep '15

Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, you really need to specify FI_ORDER_SAS for both transmit and receive if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; the MPI-style tagging interface doesn't make much sense unless the underlying fabric provider guarnatees ordering). Signed-off-by: Jeff Squyres <jsquyres(a)cisco.com> --- src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c index 5de47b7..2cf3337 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c @@ -77,10 +77,12 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ /* We expect to register all memory up front for use with this */ /* endpoint, so the netmod requires dynamic memory regions */ /* ------------------------------------------------------------------------ */ - hints = fi_allocinfo(); - hints->mode = FI_CONTEXT; - hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ - hints->caps = FI_TAGGED; /* Tag matching interface */ + hints = fi_allocinfo(); + hints->mode = FI_CONTEXT; + hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ + hints->caps = FI_TAGGED; /* Tag matching interface */ + hints->tx_attr->msg_order = FI_ORDER_SAS; + hints->rx_attr->msg_order = FI_ORDER_SAS; hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); -- 2.4.3

1 0