August 2015 - commits - mailman3.cels.anl.gov

[mpich] MPICH primary repository branch, master, updated. v3.2b4-76-gf6277db
by noreply＠mpich.org 05 Aug '15

05 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via f6277dbb69ee729c098f8ba811e4c39ad5c41bb0 (commit) from 7ab7ca15500191aaf3e38295fceefccf254cbb0f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/f6277dbb69ee729c098f8ba811e4c39ad… commit f6277dbb69ee729c098f8ba811e4c39ad5c41bb0 Author: Charles J Archer <charles.j.archer(a)intel.com> Date: Wed Jul 22 12:17:59 2015 -0700 Move fastpath static inline to appropriate location * MPIR_Request_complete_fastpath is only used by waitall.c Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/src/include/mpiimpl.h b/src/include/mpiimpl.h index 02d29c0..d314fbc 100644 --- a/src/include/mpiimpl.h +++ b/src/include/mpiimpl.h @@ -4423,39 +4423,6 @@ int MPIR_Waitall_impl(int count, MPI_Request array_of_requests[], int MPIR_Comm_set_attr_impl(MPID_Comm *comm_ptr, int comm_keyval, void *attribute_val, MPIR_AttrType attrType); - -/* The "fastpath" version of MPIR_Request_complete. It only handles - * MPID_REQUEST_SEND and MPID_REQUEST_RECV kinds, and it does not attempt to - * deal with status structures under the assumption that bleeding fast code will - * pass either MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE as appropriate. This - * routine (or some a variation of it) is an unfortunately necessary stunt to - * get high message rates on key benchmarks for high-end systems. - */ -#undef FUNCNAME -#define FUNCNAME MPIR_Request_complete_fastpath -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -static inline int MPIR_Request_complete_fastpath(MPI_Request *request, MPID_Request *request_ptr) -{ - int mpi_errno = MPI_SUCCESS; - - MPIU_Assert(request_ptr->kind == MPID_REQUEST_SEND || request_ptr->kind == MPID_REQUEST_RECV); - - if (request_ptr->kind == MPID_REQUEST_SEND) { - /* FIXME: are Ibsend requests added to the send queue? */ - MPIR_SENDQ_FORGET(request_ptr); - } - - /* the completion path for SEND and RECV is the same at this time, modulo - * the SENDQ hook above */ - mpi_errno = request_ptr->status.MPI_ERROR; - MPID_Request_release(request_ptr); - *request = MPI_REQUEST_NULL; - - /* avoid normal fn_exit/fn_fail jump pattern to reduce jumps and compiler confusion */ - return mpi_errno; -} - /* Pull the error status out of the tag space and put it into an errflag. */ #undef FUNCNAME #define FUNCNAME MPIR_process_status diff --git a/src/mpi/pt2pt/waitall.c b/src/mpi/pt2pt/waitall.c index 98cc267..15096e1 100644 --- a/src/mpi/pt2pt/waitall.c +++ b/src/mpi/pt2pt/waitall.c @@ -30,6 +30,39 @@ int MPI_Waitall(int count, MPI_Request array_of_requests[], MPI_Status array_of_ #undef MPI_Waitall #define MPI_Waitall PMPI_Waitall + +/* The "fastpath" version of MPIR_Request_complete. It only handles + * MPID_REQUEST_SEND and MPID_REQUEST_RECV kinds, and it does not attempt to + * deal with status structures under the assumption that bleeding fast code will + * pass either MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE as appropriate. This + * routine (or some a variation of it) is an unfortunately necessary stunt to + * get high message rates on key benchmarks for high-end systems. + */ +#undef FUNCNAME +#define FUNCNAME request_complete_fastpath +#undef FCNAME +#define FCNAME MPIU_QUOTE(FUNCNAME) +static inline int request_complete_fastpath(MPI_Request *request, MPID_Request *request_ptr) +{ + int mpi_errno = MPI_SUCCESS; + + MPIU_Assert(request_ptr->kind == MPID_REQUEST_SEND || request_ptr->kind == MPID_REQUEST_RECV); + + if (request_ptr->kind == MPID_REQUEST_SEND) { + /* FIXME: are Ibsend requests added to the send queue? */ + MPIR_SENDQ_FORGET(request_ptr); + } + + /* the completion path for SEND and RECV is the same at this time, modulo + * the SENDQ hook above */ + mpi_errno = request_ptr->status.MPI_ERROR; + MPID_Request_release(request_ptr); + *request = MPI_REQUEST_NULL; + + /* avoid normal fn_exit/fn_fail jump pattern to reduce jumps and compiler confusion */ + return mpi_errno; +} + #undef FUNCNAME #define FUNCNAME MPIR_Waitall_impl #undef FCNAME @@ -144,7 +177,7 @@ int MPIR_Waitall_impl(int count, MPI_Request array_of_requests[], /* --END ERROR HANDLING-- */ } } - mpi_errno = MPIR_Request_complete_fastpath(&array_of_requests[i], request_ptrs[i]); + mpi_errno = request_complete_fastpath(&array_of_requests[i], request_ptrs[i]); if (mpi_errno) MPIU_ERR_POP(mpi_errno); } ----------------------------------------------------------------------- Summary of changes: src/include/mpiimpl.h | 33 --------------------------------- src/mpi/pt2pt/waitall.c | 35 ++++++++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 34 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-75-g7ab7ca1
by noreply＠mpich.org 05 Aug '15

05 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 7ab7ca15500191aaf3e38295fceefccf254cbb0f (commit) via 19b6d44e9da545a9786a5e589a7d4ad18b31ff1d (commit) via dd87515e6b92b86c28c9c00c330e115a38856ab5 (commit) via 9ff3325d17463369b785341074aa1f319ab431bb (commit) from 8b3b041804d0b179db413a0c1047346b0b2a5cf7 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/7ab7ca15500191aaf3e38295fceefccf2… commit 7ab7ca15500191aaf3e38295fceefccf254cbb0f Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Tue Aug 4 17:19:38 2015 -0500 Delete comment that makes no sense. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 346c5ba..dc4324d 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -375,8 +375,6 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ } if (op_completed == FALSE) { - /* Poke the progress engine when next_op_to_issue is not the current OP, in - * order to make sure the issuing function is re-entrant safe. */ if (win_ptr->active_req_cnt > MPIR_CVAR_CH3_RMA_POKE_PROGRESS_REQ_THRESHOLD) { mpi_errno = poke_progress_engine(); if (mpi_errno != MPI_SUCCESS) http://git.mpich.org/mpich.git/commitdiff/19b6d44e9da545a9786a5e589a7d4ad18… commit 19b6d44e9da545a9786a5e589a7d4ad18b31ff1d Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Tue Aug 4 17:04:25 2015 -0500 Make issue_ops_target not re-entrant. Function issue_ops_target should not be re-entered, otherwise it has the risk of causing too many re-entrances and using up function stack. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 44eee91..346c5ba 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -258,6 +258,8 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta } +/* Note: we should prevent this function to be re-entrant. It has the risk of + * causing too many re-entrance and using up function stack. */ #undef FUNCNAME #define FUNCNAME issue_ops_target #undef FCNAME @@ -268,12 +270,19 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ MPIDI_RMA_Op_t *curr_op = NULL; MPIDI_CH3_Pkt_flags_t flags; int first_op = 1, mpi_errno = MPI_SUCCESS; + static int fn_reentrance_check = FALSE; + + /* this function is not reentrant. if it is invoked in a + * reentrant manner, simply exit without doing anything. */ + if (fn_reentrance_check == TRUE) + goto fn_exit; + fn_reentrance_check = TRUE; (*made_progress) = 0; if (win_ptr->num_targets_with_pending_net_ops == 0 || target == NULL || target->pending_net_ops_list_head == NULL) - goto fn_exit; + goto finish_issue; /* Issue out operations in the list. */ curr_op = target->next_op_to_issue; @@ -379,6 +388,9 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ } /* end of while loop */ + finish_issue: + fn_reentrance_check = FALSE; + fn_exit: return mpi_errno; fn_fail: http://git.mpich.org/mpich.git/commitdiff/dd87515e6b92b86c28c9c00c330e115a3… commit dd87515e6b92b86c28c9c00c330e115a38856ab5 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Tue Aug 4 11:10:49 2015 -0500 Remove the restriction of FENCE/PSCW_GRANTED state in RMA progress. Originally we prevent RMA progress to deal with FENCE/PSCW_GRANTED windows, because the state switching is happening inside RMA progress and when state is switched to FENCE/PSCW_GRANTED, operations must be issued out immediately; however, now we remove the state switching from RMA progress and do it in request completion cb, in such case, we need to remove the restriction of FENCE/PSCW_GRANTED in RMA progress, otherwise operations cannot be issued out. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 9d064e6..44eee91 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -797,9 +797,7 @@ int MPIDI_CH3I_RMA_Make_progress_global(int *made_progress) int temp_progress = 0; int is_able_to_issue = 0; - if (win_ptr->states.access_state == MPIDI_RMA_NONE || - win_ptr->states.access_state == MPIDI_RMA_FENCE_GRANTED || - win_ptr->states.access_state == MPIDI_RMA_PSCW_GRANTED) + if (win_ptr->states.access_state == MPIDI_RMA_NONE) continue; /* check and try to switch window state */ http://git.mpich.org/mpich.git/commitdiff/9ff3325d17463369b785341074aa1f319… commit 9ff3325d17463369b785341074aa1f319ab431bb Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Tue Aug 4 14:01:46 2015 -0500 Delete unnecessary condition. target->next_op_to_issue pointer is assigned to curr_op->next before if branch, so it cannot equal to curr_op. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 8efc6af..9d064e6 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -368,8 +368,7 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ if (op_completed == FALSE) { /* Poke the progress engine when next_op_to_issue is not the current OP, in * order to make sure the issuing function is re-entrant safe. */ - if (target->next_op_to_issue != curr_op && - win_ptr->active_req_cnt > MPIR_CVAR_CH3_RMA_POKE_PROGRESS_REQ_THRESHOLD) { + if (win_ptr->active_req_cnt > MPIR_CVAR_CH3_RMA_POKE_PROGRESS_REQ_THRESHOLD) { mpi_errno = poke_progress_engine(); if (mpi_errno != MPI_SUCCESS) MPIU_ERR_POP(mpi_errno); ----------------------------------------------------------------------- Summary of changes: src/mpid/ch3/src/ch3u_rma_progress.c | 23 +++++++++++++++-------- 1 files changed, 15 insertions(+), 8 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-71-g8b3b041
by noreply＠mpich.org 04 Aug '15

04 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 8b3b041804d0b179db413a0c1047346b0b2a5cf7 (commit) from d5a534c547e168b65de6bca7a52d7dbe00118596 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/8b3b041804d0b179db413a0c1047346b0… commit 8b3b041804d0b179db413a0c1047346b0b2a5cf7 Author: Huiwei Lu <huiweilu(a)mcs.anl.gov> Date: Tue Aug 4 16:44:50 2015 -0500 Revert "Mark comm_idup_comm2 as xfail" This reverts commit d5a534c547e168b65de6bca7a52d7dbe00118596. diff --git a/test/mpi/comm/comm_idup_comm2.c b/test/mpi/comm/comm_idup_comm2.c index 50e80d7..dd476dc 100644 --- a/test/mpi/comm/comm_idup_comm2.c +++ b/test/mpi/comm/comm_idup_comm2.c @@ -45,7 +45,6 @@ int main(int argc, char **argv) /* Overlap pending idups with various comm generation functions */ -#if 0 /* Comm_dup */ MPI_Comm_dup(dupcomm, &outcomm); errs += MTestTestComm(outcomm); @@ -65,7 +64,6 @@ int main(int argc, char **argv) MPI_Group_free(&high_group); errs += MTestTestComm(outcomm); MTestFreeComm(&outcomm); -#endif /* Comm_create_group, even ranks of dupcomm */ /* exclude the odd ranks */ @@ -87,7 +85,6 @@ int main(int argc, char **argv) errs += MTestTestComm(outcomm); MTestFreeComm(&outcomm); -#if 0 /* Intercomm_create & Intercomm_merge */ MPI_Comm_split(dupcomm, (rank < size / 2), rank, &local_comm); @@ -111,7 +108,6 @@ int main(int argc, char **argv) errs += MTestTestComm(outcomm); MTestFreeComm(&outcomm); -#endif MPI_Waitall(NUM_IDUPS, reqs, MPI_STATUSES_IGNORE); for (i = 0; i < NUM_IDUPS; i++) { diff --git a/test/mpi/comm/testlist b/test/mpi/comm/testlist index fce0780..416237b 100644 --- a/test/mpi/comm/testlist +++ b/test/mpi/comm/testlist @@ -33,7 +33,7 @@ comm_idup_iallreduce 6 mpiversion=3.0 comm_idup_nb 6 mpiversion=3.0 comm_idup_isend 6 mpiversion=3.0 comm_idup_comm 6 mpiversion=3.0 -comm_idup_comm2 6 mpiversion=3.0 xfail=ticket2295 +comm_idup_comm2 6 mpiversion=3.0 dup_with_info 2 mpiversion=3.0 dup_with_info 4 mpiversion=3.0 dup_with_info 9 mpiversion=3.0 ----------------------------------------------------------------------- Summary of changes: test/mpi/comm/comm_idup_comm2.c | 4 ---- test/mpi/comm/testlist | 2 +- 2 files changed, 1 insertions(+), 5 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-70-gd5a534c
by noreply＠mpich.org 04 Aug '15

04 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via d5a534c547e168b65de6bca7a52d7dbe00118596 (commit) via 7696a562268863702e6d6197092bd5468315bf9d (commit) from 1eff6db8f1dbf803c0b4a9fc1f601b1ac076bf8f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/d5a534c547e168b65de6bca7a52d7dbe0… commit d5a534c547e168b65de6bca7a52d7dbe00118596 Author: Huiwei Lu <huiweilu(a)mcs.anl.gov> Date: Tue Aug 4 15:54:29 2015 -0500 Mark comm_idup_comm2 as xfail Signed-off-by: Lena Oden <loden(a)anl.gov> diff --git a/test/mpi/comm/comm_idup_comm2.c b/test/mpi/comm/comm_idup_comm2.c index dd476dc..50e80d7 100644 --- a/test/mpi/comm/comm_idup_comm2.c +++ b/test/mpi/comm/comm_idup_comm2.c @@ -45,6 +45,7 @@ int main(int argc, char **argv) /* Overlap pending idups with various comm generation functions */ +#if 0 /* Comm_dup */ MPI_Comm_dup(dupcomm, &outcomm); errs += MTestTestComm(outcomm); @@ -64,6 +65,7 @@ int main(int argc, char **argv) MPI_Group_free(&high_group); errs += MTestTestComm(outcomm); MTestFreeComm(&outcomm); +#endif /* Comm_create_group, even ranks of dupcomm */ /* exclude the odd ranks */ @@ -85,6 +87,7 @@ int main(int argc, char **argv) errs += MTestTestComm(outcomm); MTestFreeComm(&outcomm); +#if 0 /* Intercomm_create & Intercomm_merge */ MPI_Comm_split(dupcomm, (rank < size / 2), rank, &local_comm); @@ -108,6 +111,7 @@ int main(int argc, char **argv) errs += MTestTestComm(outcomm); MTestFreeComm(&outcomm); +#endif MPI_Waitall(NUM_IDUPS, reqs, MPI_STATUSES_IGNORE); for (i = 0; i < NUM_IDUPS; i++) { diff --git a/test/mpi/comm/testlist b/test/mpi/comm/testlist index 416237b..fce0780 100644 --- a/test/mpi/comm/testlist +++ b/test/mpi/comm/testlist @@ -33,7 +33,7 @@ comm_idup_iallreduce 6 mpiversion=3.0 comm_idup_nb 6 mpiversion=3.0 comm_idup_isend 6 mpiversion=3.0 comm_idup_comm 6 mpiversion=3.0 -comm_idup_comm2 6 mpiversion=3.0 +comm_idup_comm2 6 mpiversion=3.0 xfail=ticket2295 dup_with_info 2 mpiversion=3.0 dup_with_info 4 mpiversion=3.0 dup_with_info 9 mpiversion=3.0 http://git.mpich.org/mpich.git/commitdiff/7696a562268863702e6d6197092bd5468… commit 7696a562268863702e6d6197092bd5468315bf9d Author: Huiwei Lu <huiweilu(a)mcs.anl.gov> Date: Tue Aug 4 13:38:41 2015 -0500 Adds a test for overlapping comm_dup and comm_idup The new test is similar to comm_idup_comm but has an additional step. The overlapping MPI_Comm_idup and other communicator creation functions are working on two different comms. Signed-off-by: Lena Oden <loden(a)anl.gov> diff --git a/test/mpi/comm/Makefile.am b/test/mpi/comm/Makefile.am index 105fb74..033d400 100644 --- a/test/mpi/comm/Makefile.am +++ b/test/mpi/comm/Makefile.am @@ -41,4 +41,5 @@ noinst_PROGRAMS = \ comm_idup_nb \ comm_idup_iallreduce \ comm_idup_comm \ + comm_idup_comm2 \ comm_info diff --git a/test/mpi/comm/comm_idup_comm2.c b/test/mpi/comm/comm_idup_comm2.c new file mode 100644 index 0000000..dd476dc --- /dev/null +++ b/test/mpi/comm/comm_idup_comm2.c @@ -0,0 +1,124 @@ +/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ +/* + * (C) 2015 by Argonne National Laboratory. + * See COPYRIGHT in top-level directory. + */ + +/* This test tests overlapping of Comm_idups with other comm. generations calls */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include "mpi.h" +#include "mpitest.h" + +#define NUM_IDUPS 5 + +int main(int argc, char **argv) +{ + int errs = 0; + int i; + int rank, size; + int *excl; + int ranges[1][3]; + int isLeft, rleader; + MPI_Group world_group, high_group, even_group; + MPI_Comm local_comm, inter_comm, test_comm, outcomm, dupcomm; + MPI_Comm idupcomms[NUM_IDUPS]; + MPI_Request reqs[NUM_IDUPS]; + + MTest_Init(&argc, &argv); + MPI_Comm_rank(MPI_COMM_WORLD, &rank); + MPI_Comm_size(MPI_COMM_WORLD, &size); + MPI_Comm_group(MPI_COMM_WORLD, &world_group); + MPI_Comm_dup(MPI_COMM_WORLD, &dupcomm); + + if (size < 2) { + printf("this test requires at least 2 processes\n"); + MPI_Abort(MPI_COMM_WORLD, 1); + } + + /* Idup MPI_COMM_WORLD multiple times */ + for (i = 0; i < NUM_IDUPS; i++) { + MPI_Comm_idup(MPI_COMM_WORLD, &idupcomms[i], &reqs[i]); + } + + /* Overlap pending idups with various comm generation functions */ + + /* Comm_dup */ + MPI_Comm_dup(dupcomm, &outcomm); + errs += MTestTestComm(outcomm); + MTestFreeComm(&outcomm); + + /* Comm_split */ + MPI_Comm_split(dupcomm, rank % 2, size - rank, &outcomm); + errs += MTestTestComm(outcomm); + MTestFreeComm(&outcomm); + + /* Comm_create, high half of dupcomm */ + ranges[0][0] = size / 2; + ranges[0][1] = size - 1; + ranges[0][2] = 1; + MPI_Group_range_incl(world_group, 1, ranges, &high_group); + MPI_Comm_create(dupcomm, high_group, &outcomm); + MPI_Group_free(&high_group); + errs += MTestTestComm(outcomm); + MTestFreeComm(&outcomm); + + /* Comm_create_group, even ranks of dupcomm */ + /* exclude the odd ranks */ + excl = malloc((size / 2) * sizeof(int)); + for (i = 0; i < size / 2; i++) + excl[i] = (2 * i) + 1; + + MPI_Group_excl(world_group, size / 2, excl, &even_group); + free(excl); + + if (rank % 2 == 0) { + MPI_Comm_create_group(dupcomm, even_group, 0, &outcomm); + } + else { + outcomm = MPI_COMM_NULL; + } + MPI_Group_free(&even_group); + + errs += MTestTestComm(outcomm); + MTestFreeComm(&outcomm); + + /* Intercomm_create & Intercomm_merge */ + MPI_Comm_split(dupcomm, (rank < size / 2), rank, &local_comm); + + if (rank == 0) { + rleader = size / 2; + } + else if (rank == size / 2) { + rleader = 0; + } + else { + rleader = -1; + } + isLeft = rank < size / 2; + + MPI_Intercomm_create(local_comm, 0, dupcomm, rleader, 99, &inter_comm); + MPI_Intercomm_merge(inter_comm, isLeft, &outcomm); + MPI_Comm_free(&local_comm); + + errs += MTestTestComm(inter_comm); + MTestFreeComm(&inter_comm); + + errs += MTestTestComm(outcomm); + MTestFreeComm(&outcomm); + + MPI_Waitall(NUM_IDUPS, reqs, MPI_STATUSES_IGNORE); + for (i = 0; i < NUM_IDUPS; i++) { + errs += MTestTestComm(idupcomms[i]); + MPI_Comm_free(&idupcomms[i]); + } + + MPI_Group_free(&world_group); + MPI_Comm_free(&dupcomm); + + MTest_Finalize(errs); + MPI_Finalize(); + return 0; +} diff --git a/test/mpi/comm/testlist b/test/mpi/comm/testlist index 31140cc..416237b 100644 --- a/test/mpi/comm/testlist +++ b/test/mpi/comm/testlist @@ -33,6 +33,7 @@ comm_idup_iallreduce 6 mpiversion=3.0 comm_idup_nb 6 mpiversion=3.0 comm_idup_isend 6 mpiversion=3.0 comm_idup_comm 6 mpiversion=3.0 +comm_idup_comm2 6 mpiversion=3.0 dup_with_info 2 mpiversion=3.0 dup_with_info 4 mpiversion=3.0 dup_with_info 9 mpiversion=3.0 ----------------------------------------------------------------------- Summary of changes: test/mpi/comm/Makefile.am | 1 + .../comm/{comm_idup_comm.c => comm_idup_comm2.c} | 24 ++++++++++++------- test/mpi/comm/testlist | 1 + 3 files changed, 17 insertions(+), 9 deletions(-) copy test/mpi/comm/{comm_idup_comm.c => comm_idup_comm2.c} (81%) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-68-g1eff6db
by noreply＠mpich.org 04 Aug '15

04 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 1eff6db8f1dbf803c0b4a9fc1f601b1ac076bf8f (commit) from bb707893aa7fe5d0741abd306698fe7397763c36 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/1eff6db8f1dbf803c0b4a9fc1f601b1ac… commit 1eff6db8f1dbf803c0b4a9fc1f601b1ac076bf8f Author: Jens Hauke <hauke(a)par-tec.com> Date: Mon Aug 3 11:39:43 2015 -0500 avoid dependency on GPFS if not requested Par-tec had an issue with libmpi.so depending on libgpfs even without romio configured to use it. Make the gpfs checks optional to remove the dependency. Signed-off-by: Rob Latham <robl(a)mcs.anl.gov> diff --git a/src/mpi/romio/configure.ac b/src/mpi/romio/configure.ac index 598e790..6e10c08 100644 --- a/src/mpi/romio/configure.ac +++ b/src/mpi/romio/configure.ac @@ -974,11 +974,13 @@ fi AS_IF([test -n "$file_system_gpfs"], [SYSDEP_INC=-I${prefix}/include], [SYSDEP_INC=]) -AC_CHECK_HEADERS([gpfs.h gpfs_fcntl.h]) -AS_IF([test "$ac_cv_header_gpfs_h" = "yes" -o "$ac_cv_header_gpfs_fcntl_h" = "yes"], [ - AC_SEARCH_LIBS([gpfs_fcntl], [gpfs], [], - [AC_MSG_ERROR([Library containing gpfs_fcntl symbols not found])]) -]) +AS_IF([test -n "$file_system_gpfs"], + AC_CHECK_HEADERS([gpfs.h gpfs_fcntl.h]) + AS_IF([test "$ac_cv_header_gpfs_h" = "yes" -o "$ac_cv_header_gpfs_fcntl_h" = "yes"], [ + AC_SEARCH_LIBS([gpfs_fcntl], [gpfs], [], + [AC_MSG_ERROR([Library containing gpfs_fcntl symbols not found])]) + ]) + ) # Check for presence and characteristics of async. I/O calls if # not disabled. ----------------------------------------------------------------------- Summary of changes: src/mpi/romio/configure.ac | 12 +++++++----- 1 files changed, 7 insertions(+), 5 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-67-gbb70789
by noreply＠mpich.org 04 Aug '15

04 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via bb707893aa7fe5d0741abd306698fe7397763c36 (commit) via 3e968df88eeeb8d1a7865af2ff020032a76c537b (commit) via 9d4d43f1562e2c0cd9ac42d8da3d45ce6e06d075 (commit) from 5fb28f76adbf88aa5f27fcfbe9d0a48109d7a149 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/bb707893aa7fe5d0741abd306698fe739… commit bb707893aa7fe5d0741abd306698fe7397763c36 Author: Rob Latham <robl(a)mcs.anl.gov> Date: Fri Jul 31 10:05:47 2015 -0500 configure-time check for newer PanFS features recent versions of PanFS provide a 'RADID N', instead of classic RADID0, RAID5, etc. Tested-by: Ron Victorelli <rvictorelli(a)panasas.com> No Reviewer diff --git a/src/mpi/romio/adio/ad_panfs/Makefile.mk b/src/mpi/romio/adio/ad_panfs/Makefile.mk index 68e92d5..ad412d9 100644 --- a/src/mpi/romio/adio/ad_panfs/Makefile.mk +++ b/src/mpi/romio/adio/ad_panfs/Makefile.mk @@ -5,13 +5,20 @@ ## See COPYRIGHT in top-level directory. ## +if BUILD_PANFS_OPEN6 +# override open with one that uses newer features +panfs_open = adio/ad_panfs/ad_panfs_open6.c +else +panfs_open = adio/ad_panfs/ad_panfs_open.c +endif + if BUILD_AD_PANFS noinst_HEADERS += adio/ad_panfs/ad_panfs.h romio_other_sources += \ adio/ad_panfs/ad_panfs.c \ - adio/ad_panfs/ad_panfs_open.c \ + $(panfs_open) \ adio/ad_panfs/ad_panfs_hints.c \ adio/ad_panfs/ad_panfs_read.c \ adio/ad_panfs/ad_panfs_resize.c \ diff --git a/src/mpi/romio/adio/ad_panfs/ad_panfs.c b/src/mpi/romio/adio/ad_panfs/ad_panfs.c index fc75f7d..8bc14fc 100644 --- a/src/mpi/romio/adio/ad_panfs/ad_panfs.c +++ b/src/mpi/romio/adio/ad_panfs/ad_panfs.c @@ -11,8 +11,18 @@ /* adioi.h has the ADIOI_Fns_struct define */ #include "adioi.h" +#ifndef ROMIOCONF_H_INCLUDED +#include "romioconf.h" +#define ROMIOCONF_H_INCLUDED +#endif + + struct ADIOI_Fns_struct ADIO_PANFS_operations = { - ADIOI_PANFS_Open, /* Open */ +#ifdef HAVE_PAN_FS_CLIENT_RAIDN_ENCODING_T + ADIOI_PANFS_Open6, /* Open, using newer Panasas features */ +#else + ADIOI_PANFS_Open, /* open, but using Panasas5 and earlier features */ +#endif ADIOI_GEN_OpenColl, ADIOI_PANFS_ReadContig, /* ReadContig */ ADIOI_PANFS_WriteContig, /* WriteContig */ diff --git a/src/mpi/romio/configure.ac b/src/mpi/romio/configure.ac index 5753856..598e790 100644 --- a/src/mpi/romio/configure.ac +++ b/src/mpi/romio/configure.ac @@ -792,7 +792,10 @@ if test -n "$file_system_panfs"; then AC_DEFINE(ROMIO_PANFS,1,[Define for ROMIO with PANFS]), AC_MSG_ERROR([PANFS support requested but cannot find pan_fs_client_cw_mode.h header file]) ) + AC_CHECK_TYPES([pan_fs_client_raidn_encoding_t], , , + [[#include <pan_fs_client_cw_mode.h>]]) fi +AM_CONDITIONAL([BUILD_PANFS_OPEN6], [test "X$ac_cv_type_pan_fs_client_raidn_encoding_t" = "Xyes"]) if test -n "$file_system_ufs"; then AC_DEFINE(ROMIO_UFS,1,[Define for ROMIO with UFS]) http://git.mpich.org/mpich.git/commitdiff/3e968df88eeeb8d1a7865af2ff020032a… commit 3e968df88eeeb8d1a7865af2ff020032a76c537b Author: Ron Victorelli <rvictorelli(a)pgh-octal-9-1.palab.panasas.com> Date: Fri Jul 31 16:16:10 2015 -0500 Remove RAID0 support and add RAIDN to MPICH Signed-off-by: Rob Latham <robl(a)mcs.anl.gov> diff --git a/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c b/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c index 8457793..6906ff0 100644 --- a/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c +++ b/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c @@ -32,7 +32,9 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) unsigned long int layout_parity_stripe_width = 0; unsigned long int layout_parity_stripe_depth = 0; unsigned long int layout_total_num_comps = 0; + unsigned long int layout_max_faults = 2; pan_fs_client_layout_visit_t layout_visit_policy = PAN_FS_CLIENT_LAYOUT_VISIT__ROUND_ROBIN; + pan_fs_client_raidn_encoding_t layout_encoding = PAN_FS_CLIENT_LAYOUT_RAIDN_ENCODING_RS; int myrank; MPI_Comm_rank(fd->comm, &myrank); @@ -61,33 +63,84 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) if (flag) { layout_parity_stripe_depth = strtoul(value, NULL, 10); } + ADIOI_Info_get(fd->info, "panfs_layout_max_faults", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + layout_max_faults = strtoul(value, NULL, 10); + } ADIOI_Info_get(fd->info, "panfs_layout_visit_policy", MPI_MAX_INFO_VAL, value, &flag); if (flag) { layout_visit_policy = strtoul(value, NULL, 10); } + ADIOI_Info_get(fd->info, "panfs_layout_encoding", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + layout_encoding = strtoul(value, NULL, 10); + } ADIOI_Free(value); amode = amode | O_CREAT; - /* Check for valid set of hints */ - if ((layout_type < PAN_FS_CLIENT_LAYOUT_TYPE__DEFAULT) || - (layout_type > PAN_FS_CLIENT_LAYOUT_TYPE__RAID10)) { + /* Check for valid set of hints + * + * Note that RAID0 has been dropped. In the event PAN_FS_CLIENT_LAYOUT_TYPE__RAID0 + * enumeraion no longer exists, the following check will still be correct. + * + * The enumerations looks as follows: + * + * enum pan_fs_client_layout_agg_type_e { + * PAN_FS_CLIENT_LAYOUT_TYPE__INVALID = 0, - *INVALID + * PAN_FS_CLIENT_LAYOUT_TYPE__DEFAULT = 1, - VALID + * PAN_FS_CLIENT_LAYOUT_TYPE__RAID0 = 2, - *INVALID + * PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE = 3, - VALID + * PAN_FS_CLIENT_LAYOUT_TYPE__RAID10 = 4, - VALID + * PAN_FS_CLIENT_LAYOUT_TYPE__RAIDN_PARITY_STRIPE = 5 - VALID + * }; + */ + if (((layout_type < PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) && + (layout_type != PAN_FS_CLIENT_LAYOUT_TYPE__DEFAULT)) || + (layout_type > PAN_FS_CLIENT_LAYOUT_TYPE__RAIDN_PARITY_STRIPE)) { FPRINTF(stderr, "%s: panfs_layout_type is not a valid value: %u.\n", myname, layout_type); MPI_Abort(MPI_COMM_WORLD, 1); } - if ((layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID0) && - ((layout_stripe_unit == 0) || (layout_total_num_comps == 0))) { - if (layout_stripe_unit == 0) { + if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAIDN_PARITY_STRIPE) { + if ((layout_stripe_unit == 0) || + (layout_parity_stripe_width == 0) || + (layout_parity_stripe_depth == 0) || (layout_total_num_comps == 0)) { + if (layout_stripe_unit == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_stripe_unit hint which is necessary to specify a valid RAIDN parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_total_num_comps == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_total_num_comps hint which is necessary to specify a valid RAIDN parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_parity_stripe_width == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_parity_stripe_width hint which is necessary to specify a valid RAIDN parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_parity_stripe_depth == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_parity_stripe_depth hint which is necessary to specify a valid RAIDN parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + MPI_Abort(MPI_COMM_WORLD, 1); + } + /* as of 6.0.x release, we only support max_faults == 2 */ + if (layout_max_faults != 2) { FPRINTF(stderr, - "%s: MPI_Info does not contain the panfs_layout_stripe_unit hint which is necessary to specify a valid RAID0 layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + "%s: panfs_layout_max_faults is not a valid value. Setting default of 2\n", myname); + layout_max_faults = 2; } - if (layout_total_num_comps == 0) { + /* as of 6.0.x release, we only support RS enconding */ + if (layout_encoding != PAN_FS_CLIENT_LAYOUT_RAIDN_ENCODING_RS) { FPRINTF(stderr, - "%s: MPI_Info does not contain the panfs_layout_total_num_comps hint which is necessary to specify a valid RAID0 layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", - myname); + "%s: panfs_layout_encoding is not a valid value: %u. Setting to default of %u\n", + myname, layout_encoding, PAN_FS_CLIENT_LAYOUT_RAIDN_ENCODING_RS); + layout_encoding = PAN_FS_CLIENT_LAYOUT_RAIDN_ENCODING_RS; } - MPI_Abort(MPI_COMM_WORLD, 1); } if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) { if ((layout_stripe_unit == 0) || @@ -149,9 +202,9 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) * already optimizes performance by only calling this function with * ADIO_CREATE on rank 0. Therefore, we don't need to worry about * implementing that optimization here. */ - if ((layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID0) || - (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) - || (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID10)) { + if ((layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) || + (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID10) || + (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAIDN_PARITY_STRIPE)) { pan_fs_client_layout_create_args_t file_create_args; int fd_dir; char *slash; @@ -173,10 +226,13 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) MPI_Abort(MPI_COMM_WORLD, 1); } else if (err == 0) { + /* ensure that we have the same semantics here and in the call to creat(). In the latter, we do not + * use O_EXCL so a create on an existing file should not fail. + */ FPRINTF(stderr, - "%s: Cannot create PanFS file with ioctl when file already exists.\n", + "%s: Cannot create PanFS file with ioctl when file already exists, using open() syscall.\n", myname); - MPI_Abort(MPI_COMM_WORLD, 1); + goto use_open_syscall; } else { /* (err == -1) && (errno == ENOENT) */ @@ -216,7 +272,20 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) strlen(fd->filename) + 1); file_create_args.layout.agg_type = layout_type; file_create_args.layout.layout_is_valid = 1; - if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) { + if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAIDN_PARITY_STRIPE) { + file_create_args.layout.u.raidn_parity_stripe.total_num_comps = + layout_total_num_comps; + file_create_args.layout.u.raidn_parity_stripe.parity_stripe_width = + layout_parity_stripe_width; + file_create_args.layout.u.raidn_parity_stripe.parity_stripe_depth = + layout_parity_stripe_depth; + file_create_args.layout.u.raidn_parity_stripe.stripe_unit = + layout_stripe_unit; + file_create_args.layout.u.raidn_parity_stripe.max_faults = + layout_max_faults; + file_create_args.layout.u.raidn_parity_stripe.encoding = layout_encoding; + } + else if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) { file_create_args.layout.u.raid1_5_parity_stripe.total_num_comps = layout_total_num_comps; file_create_args.layout.u.raid1_5_parity_stripe.parity_stripe_width = @@ -228,10 +297,6 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) file_create_args.layout.u.raid1_5_parity_stripe.layout_visit_policy = layout_visit_policy; } - else if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID0) { - file_create_args.layout.u.raid0.total_num_comps = layout_total_num_comps; - file_create_args.layout.u.raid0.stripe_unit = layout_stripe_unit; - } else if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID10) { file_create_args.layout.u.raid10.total_num_comps = layout_total_num_comps; file_create_args.layout.u.raid10.stripe_unit = layout_stripe_unit; @@ -250,6 +315,7 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) } } else { + use_open_syscall:; int create_fd = open(fd->filename, amode, perm); if (create_fd != -1) { close(create_fd); @@ -299,13 +365,27 @@ void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) ADIOI_Info_set(fd->info, "panfs_layout_type", temp_buffer); if (file_query_args.layout.layout_is_valid == 1) { switch (file_query_args.layout.agg_type) { - case PAN_FS_CLIENT_LAYOUT_TYPE__RAID0: + case PAN_FS_CLIENT_LAYOUT_TYPE__RAIDN_PARITY_STRIPE: ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", - file_query_args.layout.u.raid0.stripe_unit); + file_query_args.layout.u.raidn_parity_stripe.stripe_unit); ADIOI_Info_set(fd->info, "panfs_layout_stripe_unit", temp_buffer); ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", - file_query_args.layout.u.raid0.total_num_comps); + file_query_args.layout.u.raidn_parity_stripe. + parity_stripe_width); + ADIOI_Info_set(fd->info, "panfs_layout_parity_stripe_width", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raidn_parity_stripe. + parity_stripe_depth); + ADIOI_Info_set(fd->info, "panfs_layout_parity_stripe_depth", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raidn_parity_stripe.total_num_comps); ADIOI_Info_set(fd->info, "panfs_layout_total_num_comps", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raidn_parity_stripe.max_faults); + ADIOI_Info_set(fd->info, "panfs_layout_max_faults", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raidn_parity_stripe.encoding); + ADIOI_Info_set(fd->info, "panfs_layout_encoding", temp_buffer); break; case PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE: ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", http://git.mpich.org/mpich.git/commitdiff/9d4d43f1562e2c0cd9ac42d8da3d45ce6… commit 9d4d43f1562e2c0cd9ac42d8da3d45ce6e06d075 Author: Rob Latham <robl(a)mcs.anl.gov> Date: Fri Jul 31 16:09:32 2015 -0500 Laying foundation for new features for Panasas New versions of Panasas (Panfs-6 and up) implement RAID0 and RAID5 in terms of newer RAIDN support, but Panfs-5 and older don't know what RAIDN is. This new 'open6' will in subsequent patches make use of new features, but we'll keep the old open around in case sites have not upgraded. diff --git a/src/mpi/romio/adio/ad_panfs/ad_panfs.h b/src/mpi/romio/adio/ad_panfs/ad_panfs.h index 0f2e9f5..31a0f3d 100644 --- a/src/mpi/romio/adio/ad_panfs/ad_panfs.h +++ b/src/mpi/romio/adio/ad_panfs/ad_panfs.h @@ -32,6 +32,8 @@ #endif void ADIOI_PANFS_Open(ADIO_File fd, int *error_code); +/* Panasas 6 introduced some new features */ +void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code); void ADIOI_PANFS_SetInfo(ADIO_File fd, MPI_Info users_info, int *error_code); void ADIOI_PANFS_ReadContig(ADIO_File fd, void *buf, int count, MPI_Datatype datatype, int file_ptr_type, diff --git a/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c b/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c new file mode 100644 index 0000000..8457793 --- /dev/null +++ b/src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c @@ -0,0 +1,360 @@ +/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ +/* + * ad_panfs_open.c + * + * Copyright (C) 2001 University of Chicago. + * See COPYRIGHT notice in top-level directory. + */ + +#include "ad_panfs.h" +#include <string.h> +#include <pan_fs_client_cw_mode.h> +#define TEMP_BUFFER_SIZE 64 + +void ADIOI_PANFS_Open6(ADIO_File fd, int *error_code) +{ + char *value; + int perm, old_mask, amode, flag; + static char myname[] = "ADIOI_PANFS_OPEN6"; + + if (fd->perm == ADIO_PERM_NULL) { + old_mask = umask(022); + umask(old_mask); + perm = ~old_mask & 0666; + } + else + perm = fd->perm; + + amode = 0; + if (fd->access_mode & ADIO_CREATE) { + pan_fs_client_layout_agg_type_t layout_type = PAN_FS_CLIENT_LAYOUT_TYPE__DEFAULT; + unsigned long int layout_stripe_unit = 0; + unsigned long int layout_parity_stripe_width = 0; + unsigned long int layout_parity_stripe_depth = 0; + unsigned long int layout_total_num_comps = 0; + pan_fs_client_layout_visit_t layout_visit_policy = PAN_FS_CLIENT_LAYOUT_VISIT__ROUND_ROBIN; + int myrank; + + MPI_Comm_rank(fd->comm, &myrank); + + *error_code = MPI_SUCCESS; + value = (char *) ADIOI_Malloc((MPI_MAX_INFO_VAL + 1) * sizeof(char)); + ADIOI_Info_get(fd->info, "panfs_layout_type", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + layout_type = strtoul(value, NULL, 10); + } + ADIOI_Info_get(fd->info, "panfs_layout_stripe_unit", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + layout_stripe_unit = strtoul(value, NULL, 10); + } + ADIOI_Info_get(fd->info, "panfs_layout_total_num_comps", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + layout_total_num_comps = strtoul(value, NULL, 10); + } + ADIOI_Info_get(fd->info, "panfs_layout_parity_stripe_width", MPI_MAX_INFO_VAL, + value, &flag); + if (flag) { + layout_parity_stripe_width = strtoul(value, NULL, 10); + } + ADIOI_Info_get(fd->info, "panfs_layout_parity_stripe_depth", MPI_MAX_INFO_VAL, + value, &flag); + if (flag) { + layout_parity_stripe_depth = strtoul(value, NULL, 10); + } + ADIOI_Info_get(fd->info, "panfs_layout_visit_policy", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + layout_visit_policy = strtoul(value, NULL, 10); + } + ADIOI_Free(value); + + amode = amode | O_CREAT; + /* Check for valid set of hints */ + if ((layout_type < PAN_FS_CLIENT_LAYOUT_TYPE__DEFAULT) || + (layout_type > PAN_FS_CLIENT_LAYOUT_TYPE__RAID10)) { + FPRINTF(stderr, "%s: panfs_layout_type is not a valid value: %u.\n", myname, + layout_type); + MPI_Abort(MPI_COMM_WORLD, 1); + } + if ((layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID0) && + ((layout_stripe_unit == 0) || (layout_total_num_comps == 0))) { + if (layout_stripe_unit == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_stripe_unit hint which is necessary to specify a valid RAID0 layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_total_num_comps == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_total_num_comps hint which is necessary to specify a valid RAID0 layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + MPI_Abort(MPI_COMM_WORLD, 1); + } + if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) { + if ((layout_stripe_unit == 0) || + (layout_parity_stripe_width == 0) || + (layout_parity_stripe_depth == 0) || (layout_total_num_comps == 0)) { + if (layout_stripe_unit == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_stripe_unit hint which is necessary to specify a valid RAID5 parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_total_num_comps == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_total_num_comps hint which is necessary to specify a valid RAID5 parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_parity_stripe_width == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_parity_stripe_width hint which is necessary to specify a valid RAID5 parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_parity_stripe_depth == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_parity_stripe_depth hint which is necessary to specify a valid RAID5 parity stripe layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + MPI_Abort(MPI_COMM_WORLD, 1); + } + if ((layout_visit_policy < PAN_FS_CLIENT_LAYOUT_VISIT__ROUND_ROBIN) || + (layout_visit_policy > PAN_FS_CLIENT_LAYOUT_VISIT__ROUND_ROBIN_WITH_HASHED_OFFSET)) + { + FPRINTF(stderr, "%s: panfs_layout_visit_policy is not a valid value: %u.\n", myname, + layout_visit_policy); + MPI_Abort(MPI_COMM_WORLD, 1); + } + } + if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID10) { + if ((layout_stripe_unit == 0) || (layout_total_num_comps == 0)) { + if (layout_stripe_unit == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_stripe_unit hint which is necessary to specify a valid RAID10 layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + if (layout_total_num_comps == 0) { + FPRINTF(stderr, + "%s: MPI_Info does not contain the panfs_layout_total_num_comps hint which is necessary to specify a valid RAID10 layout to the PAN_FS_CLIENT_LAYOUT_CREATE_FILE ioctl.\n", + myname); + } + MPI_Abort(MPI_COMM_WORLD, 1); + } + if ((layout_visit_policy < PAN_FS_CLIENT_LAYOUT_VISIT__ROUND_ROBIN) || + (layout_visit_policy > PAN_FS_CLIENT_LAYOUT_VISIT__ROUND_ROBIN_WITH_HASHED_OFFSET)) + { + FPRINTF(stderr, "%s: panfs_layout_visit_policy is not a valid value: %u.\n", myname, + layout_visit_policy); + MPI_Abort(MPI_COMM_WORLD, 1); + } + } + /* Create the file via ioctl() or open(). ADIOI_PANFS_Open's caller + * already optimizes performance by only calling this function with + * ADIO_CREATE on rank 0. Therefore, we don't need to worry about + * implementing that optimization here. */ + if ((layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID0) || + (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) + || (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID10)) { + pan_fs_client_layout_create_args_t file_create_args; + int fd_dir; + char *slash; + struct stat stat_buf; + int err; + char *path; + + /* Check that the file does not exist before + * trying to create it. The ioctl itself should + * be able to handle this condition. Currently, + * the ioctl will return successfully if the file + * has been previously created. Filed bug 33862 + * to track the problem. + */ + err = stat(fd->filename, &stat_buf); + if ((err == -1) && (errno != ENOENT)) { + FPRINTF(stderr, "%s: Unexpected I/O Error calling stat() on PanFS file: %s.\n", + myname, strerror(errno)); + MPI_Abort(MPI_COMM_WORLD, 1); + } + else if (err == 0) { + FPRINTF(stderr, + "%s: Cannot create PanFS file with ioctl when file already exists.\n", + myname); + MPI_Abort(MPI_COMM_WORLD, 1); + } + else { + /* (err == -1) && (errno == ENOENT) */ + /* File does not exist */ + path = ADIOI_Strdup(fd->filename); + slash = strrchr(path, '/'); + if (!slash) + ADIOI_Strncpy(path, ".", 2); + else { + if (slash == path) + *(path + 1) = '\0'; + else + *slash = '\0'; + } + + /* create PanFS object */ + memset(&file_create_args, 0, sizeof(pan_fs_client_layout_create_args_t)); + /* open directory */ + fd_dir = open(path, O_RDONLY); + if (fd_dir < 0) { + FPRINTF(stderr, + "%s: I/O Error opening parent directory to create PanFS file using ioctl: %s.\n", + myname, strerror(errno)); + MPI_Abort(MPI_COMM_WORLD, 1); + } + else { + char *file_name_ptr = fd->filename; + slash = strrchr(fd->filename, '/'); + if (slash) { + file_name_ptr = slash + 1; + } + /* create file in the directory */ + file_create_args.mode = perm; + file_create_args.version = PAN_FS_CLIENT_LAYOUT_VERSION; + file_create_args.flags = PAN_FS_CLIENT_LAYOUT_CREATE_F__NONE; + ADIOI_Strncpy(file_create_args.filename, file_name_ptr, + strlen(fd->filename) + 1); + file_create_args.layout.agg_type = layout_type; + file_create_args.layout.layout_is_valid = 1; + if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE) { + file_create_args.layout.u.raid1_5_parity_stripe.total_num_comps = + layout_total_num_comps; + file_create_args.layout.u.raid1_5_parity_stripe.parity_stripe_width = + layout_parity_stripe_width; + file_create_args.layout.u.raid1_5_parity_stripe.parity_stripe_depth = + layout_parity_stripe_depth; + file_create_args.layout.u.raid1_5_parity_stripe.stripe_unit = + layout_stripe_unit; + file_create_args.layout.u.raid1_5_parity_stripe.layout_visit_policy = + layout_visit_policy; + } + else if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID0) { + file_create_args.layout.u.raid0.total_num_comps = layout_total_num_comps; + file_create_args.layout.u.raid0.stripe_unit = layout_stripe_unit; + } + else if (layout_type == PAN_FS_CLIENT_LAYOUT_TYPE__RAID10) { + file_create_args.layout.u.raid10.total_num_comps = layout_total_num_comps; + file_create_args.layout.u.raid10.stripe_unit = layout_stripe_unit; + file_create_args.layout.u.raid10.layout_visit_policy = layout_visit_policy; + } + err = ioctl(fd_dir, PAN_FS_CLIENT_LAYOUT_CREATE_FILE, &file_create_args); + if (err < 0) { + FPRINTF(stderr, + "%s: I/O Error doing ioctl on parent directory to create PanFS file using ioctl: %s.\n", + myname, strerror(errno)); + MPI_Abort(MPI_COMM_WORLD, 1); + } + err = close(fd_dir); + } + ADIOI_Free(path); + } + } + else { + int create_fd = open(fd->filename, amode, perm); + if (create_fd != -1) { + close(create_fd); + } + else { + FPRINTF(stderr, "%s: I/O Error creating PanFS file using open: %s.\n", myname, + strerror(errno)); + MPI_Abort(MPI_COMM_WORLD, 1); + } + } + } + if (fd->access_mode & ADIO_RDONLY) + amode = amode | O_RDONLY; + if (fd->access_mode & ADIO_WRONLY) + amode = amode | O_WRONLY; + if (fd->access_mode & ADIO_RDWR) + amode = amode | O_RDWR; + if (fd->access_mode & ADIO_EXCL) + amode = amode | O_EXCL; + + value = (char *) ADIOI_Malloc((MPI_MAX_INFO_VAL + 1) * sizeof(char)); + ADIOI_Info_get(fd->info, "panfs_concurrent_write", MPI_MAX_INFO_VAL, value, &flag); + if (flag) { + unsigned long int concurrent_write = strtoul(value, NULL, 10); + if (concurrent_write == 1) { + amode = amode | O_CONCURRENT_WRITE; + } + } + ADIOI_Free(value); + + fd->fd_sys = open(fd->filename, amode, perm); + fd->fd_direct = -1; + + if (fd->fd_sys != -1) { + int rc; + char temp_buffer[TEMP_BUFFER_SIZE]; + pan_fs_client_layout_query_args_t file_query_args; + memset(&file_query_args, 0, sizeof(pan_fs_client_layout_query_args_t)); + file_query_args.version = PAN_FS_CLIENT_LAYOUT_VERSION; + rc = ioctl(fd->fd_sys, PAN_FS_CLIENT_LAYOUT_QUERY_FILE, &file_query_args); + if (rc < 0) { + /* Error - set layout type to unknown */ + ADIOI_Info_set(fd->info, "panfs_layout_type", "PAN_FS_CLIENT_LAYOUT_TYPE__INVALID"); + } + else { + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", file_query_args.layout.agg_type); + ADIOI_Info_set(fd->info, "panfs_layout_type", temp_buffer); + if (file_query_args.layout.layout_is_valid == 1) { + switch (file_query_args.layout.agg_type) { + case PAN_FS_CLIENT_LAYOUT_TYPE__RAID0: + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid0.stripe_unit); + ADIOI_Info_set(fd->info, "panfs_layout_stripe_unit", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid0.total_num_comps); + ADIOI_Info_set(fd->info, "panfs_layout_total_num_comps", temp_buffer); + break; + case PAN_FS_CLIENT_LAYOUT_TYPE__RAID1_5_PARITY_STRIPE: + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid1_5_parity_stripe.stripe_unit); + ADIOI_Info_set(fd->info, "panfs_layout_stripe_unit", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid1_5_parity_stripe. + parity_stripe_width); + ADIOI_Info_set(fd->info, "panfs_layout_parity_stripe_width", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid1_5_parity_stripe. + parity_stripe_depth); + ADIOI_Info_set(fd->info, "panfs_layout_parity_stripe_depth", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid1_5_parity_stripe.total_num_comps); + ADIOI_Info_set(fd->info, "panfs_layout_total_num_comps", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid1_5_parity_stripe. + layout_visit_policy); + ADIOI_Info_set(fd->info, "panfs_layout_visit_policy", temp_buffer); + break; + case PAN_FS_CLIENT_LAYOUT_TYPE__RAID10: + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid10.stripe_unit); + ADIOI_Info_set(fd->info, "panfs_layout_stripe_unit", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid10.total_num_comps); + ADIOI_Info_set(fd->info, "panfs_layout_total_num_comps", temp_buffer); + ADIOI_Snprintf(temp_buffer, TEMP_BUFFER_SIZE, "%u", + file_query_args.layout.u.raid10.layout_visit_policy); + ADIOI_Info_set(fd->info, "panfs_layout_visit_policy", temp_buffer); + break; + case PAN_FS_CLIENT_LAYOUT_TYPE__INVALID: + case PAN_FS_CLIENT_LAYOUT_TYPE__DEFAULT: + MPI_Info_set(fd->info, "panfs_layout_type", + "PAN_FS_CLIENT_LAYOUT_TYPE__INVALID"); + default: + break; + } + } + } + } + + if ((fd->fd_sys != -1) && (fd->access_mode & ADIO_APPEND)) + fd->fp_ind = fd->fp_sys_posn = lseek(fd->fd_sys, 0, SEEK_END); + + if (fd->fd_sys == -1) { + *error_code = ADIOI_Err_create_code(myname, fd->filename, errno); + } + else + *error_code = MPI_SUCCESS; +} ----------------------------------------------------------------------- Summary of changes: src/mpi/romio/adio/ad_panfs/Makefile.mk | 9 +- src/mpi/romio/adio/ad_panfs/ad_panfs.c | 12 +- src/mpi/romio/adio/ad_panfs/ad_panfs.h | 2 + src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c | 440 ++++++++++++++++++++++++++ src/mpi/romio/configure.ac | 3 + 5 files changed, 464 insertions(+), 2 deletions(-) create mode 100644 src/mpi/romio/adio/ad_panfs/ad_panfs_open6.c hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-64-g5fb28f7
by noreply＠mpich.org 04 Aug '15

04 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 5fb28f76adbf88aa5f27fcfbe9d0a48109d7a149 (commit) via 64448061554eec5831ef4aaed2d11c58f899f5ee (commit) via 61cedc328e0b932172329fcb1d9e8539b7a7dd45 (commit) from 0bf19e5964a1436b64a5408e25a84a9c84c57c10 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/5fb28f76adbf88aa5f27fcfbe9d0a4810… commit 5fb28f76adbf88aa5f27fcfbe9d0a48109d7a149 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 23:57:41 2015 -0500 Delete redundant function definitions. MPIDI_CH3I_RMA_Ops_append and MPIDI_CH3I_RMA_Ops_unlink are essentially MPL_DL_APPEND and MPL_DL_DELETE. Here we delete them and directly use MPL macros. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpid_rma_oplist.h b/src/mpid/ch3/include/mpid_rma_oplist.h index 1702f45..4961571 100644 --- a/src/mpid/ch3/include/mpid_rma_oplist.h +++ b/src/mpid/ch3/include/mpid_rma_oplist.h @@ -593,36 +593,6 @@ static inline int MPIDI_CH3I_Win_get_op(MPID_Win * win_ptr, MPIDI_RMA_Op_t ** e) } -/* Append an element to the tail of the RMA ops list - * - * @param IN list Pointer to the RMA ops list - * @param IN elem Pointer to the element to be appended - */ -#undef FUNCNAME -#define FUNCNAME MPIDI_CH3I_RMA_Ops_append -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -static inline void MPIDI_CH3I_RMA_Ops_append(MPIDI_RMA_Ops_list_t * list, MPIDI_RMA_Op_t * elem) -{ - MPL_DL_APPEND(*list, elem); -} - - -/* Unlink an element from the RMA ops list - * - * @param IN list Pointer to the RMA ops list - * @param IN elem Pointer to the element to be unlinked - */ -#undef FUNCNAME -#define FUNCNAME MPIDI_CH3I_RMA_Ops_unlink -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -static inline void MPIDI_CH3I_RMA_Ops_unlink(MPIDI_RMA_Ops_list_t * list, MPIDI_RMA_Op_t * elem) -{ - MPL_DL_DELETE(*list, elem); -} - - /* Free an element in the RMA operations list. * * @param IN list Pointer to the RMA ops list diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index f2d1988..05d563a 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -705,8 +705,8 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar MPIU_ERR_POP(mpi_errno); } - MPIDI_CH3I_RMA_Ops_unlink(&(target->pending_net_ops_list_head), rma_op); - MPIDI_CH3I_RMA_Ops_append(list_ptr, rma_op); + MPL_DL_DELETE(target->pending_net_ops_list_head, rma_op); + MPL_DL_APPEND((*list_ptr), rma_op); win_ptr->active_req_cnt += rma_op->ref_cnt; } http://git.mpich.org/mpich.git/commitdiff/64448061554eec5831ef4aaed2d11c58f… commit 64448061554eec5831ef4aaed2d11c58f899f5ee Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 23:32:35 2015 -0500 Release the request after modifying the attributes of it. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index 408e724..f2d1988 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -671,7 +671,6 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar (*req) = NULL; } else { - MPID_Request_release((*req)); (*req)->request_completed_cb = MPIDI_CH3_Req_handler_rma_op_complete; (*req)->dev.source_win_handle = win_ptr->handle; (*req)->dev.rma_op_ptr = rma_op; @@ -682,6 +681,8 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar MPID_cc_set(&(rma_op->ureq->cc), rma_op->ref_cnt); (*req)->dev.request_handle = rma_op->ureq->handle; } + + MPID_Request_release((*req)); } } http://git.mpich.org/mpich.git/commitdiff/61cedc328e0b932172329fcb1d9e8539b… commit 61cedc328e0b932172329fcb1d9e8539b7a7dd45 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 23:31:50 2015 -0500 Delete useless if branch. Condition in if branch and following for loop are redundant. Here we delete the if branch. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index 3b5a70c..408e724 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -657,32 +657,30 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar (*op_completed) = FALSE; - if (rma_op->reqs_size > 0) { - for (i = 0; i < rma_op->reqs_size; i++) { - if (rma_op->reqs_size == 1) - req = &(rma_op->single_req); - else - req = &(rma_op->multi_reqs[i]); - - if ((*req) == NULL) - continue; + for (i = 0; i < rma_op->reqs_size; i++) { + if (rma_op->reqs_size == 1) + req = &(rma_op->single_req); + else + req = &(rma_op->multi_reqs[i]); - if (MPID_Request_is_complete((*req))) { - MPID_Request_release((*req)); - (*req) = NULL; - } - else { - MPID_Request_release((*req)); - (*req)->request_completed_cb = MPIDI_CH3_Req_handler_rma_op_complete; - (*req)->dev.source_win_handle = win_ptr->handle; - (*req)->dev.rma_op_ptr = rma_op; + if ((*req) == NULL) + continue; - rma_op->ref_cnt++; + if (MPID_Request_is_complete((*req))) { + MPID_Request_release((*req)); + (*req) = NULL; + } + else { + MPID_Request_release((*req)); + (*req)->request_completed_cb = MPIDI_CH3_Req_handler_rma_op_complete; + (*req)->dev.source_win_handle = win_ptr->handle; + (*req)->dev.rma_op_ptr = rma_op; - if (rma_op->ureq != NULL) { - MPID_cc_set(&(rma_op->ureq->cc), rma_op->ref_cnt); - (*req)->dev.request_handle = rma_op->ureq->handle; - } + rma_op->ref_cnt++; + + if (rma_op->ureq != NULL) { + MPID_cc_set(&(rma_op->ureq->cc), rma_op->ref_cnt); + (*req)->dev.request_handle = rma_op->ureq->handle; } } } ----------------------------------------------------------------------- Summary of changes: src/mpid/ch3/include/mpid_rma_oplist.h | 30 ------------------ src/mpid/ch3/include/mpidrma.h | 51 +++++++++++++++---------------- 2 files changed, 25 insertions(+), 56 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-61-g0bf19e5
by noreply＠mpich.org 04 Aug '15

04 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 0bf19e5964a1436b64a5408e25a84a9c84c57c10 (commit) via d6736337379e54a38e3ddbb513c6a6de99085d93 (commit) via 16da069598460e9482bbce66ed108c0ebed980f9 (commit) via 277290f386a2c08f8ff08f3c28e376dbecb84579 (commit) via e14a94eed62a1db783020023aed712c558620cdb (commit) from d0372e05410f30a9ea260609325417ec58479905 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/0bf19e5964a1436b64a5408e25a84a9c8… commit 0bf19e5964a1436b64a5408e25a84a9c84c57c10 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 22:30:46 2015 -0500 Bug-fix: call check_and_set_req_completion for OP piggybacked with lock. For op piggybacked with LOCK, after lock granted ACK is received, origin side should not complete and delete the operation immediately, because data in GET-like operations may not be received completely. Origin side should set req cb function to let the request to complete the operation. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index bb9d51f..3b5a70c 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -734,6 +734,7 @@ static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, MPIDI_RMA_Target_t *target = NULL; MPIDI_RMA_Op_t *op = NULL; MPIDI_CH3_Pkt_flags_t op_flags = MPIDI_CH3_PKT_FLAG_NONE; + int op_completed ATTRIBUTE((unused)) = FALSE; int mpi_errno = MPI_SUCCESS; mpi_errno = MPIDI_CH3I_Win_find_target(win_ptr, target_rank, &target); @@ -752,18 +753,6 @@ static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, if (flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_GRANTED) { - if (op->reqs_size > 0) { - MPID_Request **req = NULL; - if (op->reqs_size == 1) - req = &(op->single_req); - else - req = &(op->multi_reqs[0]); - - MPIU_Assert((*req) != NULL); - MPID_Request_release((*req)); - (*req) = NULL; - } - if ((op->pkt.type == MPIDI_CH3_PKT_ACCUMULATE || op->pkt.type == MPIDI_CH3_PKT_GET_ACCUM) && op->issued_stream_count != ALL_STREAM_UNITS_ISSUED) { /* Now we successfully issue out the first stream unit, @@ -783,23 +772,7 @@ static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, } } - if (op->ureq != NULL) { - /* Complete user request and release the ch3 ref */ - mpi_errno = MPID_Request_complete(op->ureq); - if (mpi_errno != MPI_SUCCESS) { - MPIU_ERR_POP(mpi_errno); - } - } - - MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_net_ops_list_head), op); - - if (target->pending_net_ops_list_head == NULL) { - win_ptr->num_targets_with_pending_net_ops--; - MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); - if (win_ptr->num_targets_with_pending_net_ops == 0) { - MPIDI_CH3I_Win_set_inactive(win_ptr); - } - } + check_and_set_req_completion(win_ptr, target, op, &op_completed); } else if (flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_QUEUED_DATA_DISCARDED || flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_DISCARDED) { http://git.mpich.org/mpich.git/commitdiff/d6736337379e54a38e3ddbb513c6a6de9… commit d6736337379e54a38e3ddbb513c6a6de99085d93 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 22:30:11 2015 -0500 move static function check_and_set_req_completion to header file. This static function checks the completion of requests in RMA operation and sets completion cb function if requests are not completed. Originally it was only used in RMA issuing progress, now we need to use it in handle_lock_with_op as well, so here we move it to the header file. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index 91816f0..bb9d51f 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -641,6 +641,93 @@ static inline int handle_lock_ack(MPID_Win * win_ptr, int target_rank, MPIDI_CH3 goto fn_exit; } + +#undef FUNCNAME +#define FUNCNAME check_and_set_req_completion +#undef FCNAME +#define FCNAME MPIU_QUOTE(FUNCNAME) +static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Target_t * target, + MPIDI_RMA_Op_t * rma_op, int *op_completed) +{ + int i, mpi_errno = MPI_SUCCESS; + MPID_Request **req = NULL; + MPIDI_STATE_DECL(MPID_STATE_CHECK_AND_SET_REQ_COMPLETION); + + MPIDI_RMA_FUNC_ENTER(MPID_STATE_CHECK_AND_SET_REQ_COMPLETION); + + (*op_completed) = FALSE; + + if (rma_op->reqs_size > 0) { + for (i = 0; i < rma_op->reqs_size; i++) { + if (rma_op->reqs_size == 1) + req = &(rma_op->single_req); + else + req = &(rma_op->multi_reqs[i]); + + if ((*req) == NULL) + continue; + + if (MPID_Request_is_complete((*req))) { + MPID_Request_release((*req)); + (*req) = NULL; + } + else { + MPID_Request_release((*req)); + (*req)->request_completed_cb = MPIDI_CH3_Req_handler_rma_op_complete; + (*req)->dev.source_win_handle = win_ptr->handle; + (*req)->dev.rma_op_ptr = rma_op; + + rma_op->ref_cnt++; + + if (rma_op->ureq != NULL) { + MPID_cc_set(&(rma_op->ureq->cc), rma_op->ref_cnt); + (*req)->dev.request_handle = rma_op->ureq->handle; + } + } + } + } + + if (rma_op->ref_cnt == 0) { + if (rma_op->ureq != NULL) { + mpi_errno = MPID_Request_complete(rma_op->ureq); + if (mpi_errno != MPI_SUCCESS) { + MPIU_ERR_POP(mpi_errno); + } + } + MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_net_ops_list_head), rma_op); + + (*op_completed) = TRUE; + } + else { + MPIDI_RMA_Op_t **list_ptr = NULL; + + MPIDI_CH3I_RMA_Get_issued_list_ptr(target, rma_op, list_ptr, mpi_errno); + if (mpi_errno != MPI_SUCCESS) { + MPIU_ERR_POP(mpi_errno); + } + + MPIDI_CH3I_RMA_Ops_unlink(&(target->pending_net_ops_list_head), rma_op); + MPIDI_CH3I_RMA_Ops_append(list_ptr, rma_op); + + win_ptr->active_req_cnt += rma_op->ref_cnt; + } + + if (target->pending_net_ops_list_head == NULL) { + win_ptr->num_targets_with_pending_net_ops--; + MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); + if (win_ptr->num_targets_with_pending_net_ops == 0) { + MPIDI_CH3I_Win_set_inactive(win_ptr); + } + } + + fn_exit: + MPIDI_RMA_FUNC_EXIT(MPID_STATE_CHECK_AND_SET_REQ_COMPLETION); + return mpi_errno; + fn_fail: + goto fn_exit; +} + + static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, int target_rank, MPIDI_CH3_Pkt_flags_t flags) { diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 54c5bc5..8efc6af 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -259,92 +259,6 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta #undef FUNCNAME -#define FUNCNAME check_and_set_req_completion -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Target_t * target, - MPIDI_RMA_Op_t * rma_op, int *op_completed) -{ - int i, mpi_errno = MPI_SUCCESS; - MPID_Request **req = NULL; - MPIDI_STATE_DECL(MPID_STATE_CHECK_AND_SET_REQ_COMPLETION); - - MPIDI_RMA_FUNC_ENTER(MPID_STATE_CHECK_AND_SET_REQ_COMPLETION); - - (*op_completed) = FALSE; - - if (rma_op->reqs_size > 0) { - for (i = 0; i < rma_op->reqs_size; i++) { - if (rma_op->reqs_size == 1) - req = &(rma_op->single_req); - else - req = &(rma_op->multi_reqs[i]); - - if ((*req) == NULL) - continue; - - if (MPID_Request_is_complete((*req))) { - MPID_Request_release((*req)); - (*req) = NULL; - } - else { - MPID_Request_release((*req)); - (*req)->request_completed_cb = MPIDI_CH3_Req_handler_rma_op_complete; - (*req)->dev.source_win_handle = win_ptr->handle; - (*req)->dev.rma_op_ptr = rma_op; - - rma_op->ref_cnt++; - - if (rma_op->ureq != NULL) { - MPID_cc_set(&(rma_op->ureq->cc), rma_op->ref_cnt); - (*req)->dev.request_handle = rma_op->ureq->handle; - } - } - } - } - - if (rma_op->ref_cnt == 0) { - if (rma_op->ureq != NULL) { - mpi_errno = MPID_Request_complete(rma_op->ureq); - if (mpi_errno != MPI_SUCCESS) { - MPIU_ERR_POP(mpi_errno); - } - } - MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_net_ops_list_head), rma_op); - - (*op_completed) = TRUE; - } - else { - MPIDI_RMA_Op_t **list_ptr = NULL; - - MPIDI_CH3I_RMA_Get_issued_list_ptr(target, rma_op, list_ptr, mpi_errno); - if (mpi_errno != MPI_SUCCESS) { - MPIU_ERR_POP(mpi_errno); - } - - MPIDI_CH3I_RMA_Ops_unlink(&(target->pending_net_ops_list_head), rma_op); - MPIDI_CH3I_RMA_Ops_append(list_ptr, rma_op); - - win_ptr->active_req_cnt += rma_op->ref_cnt; - } - - if (target->pending_net_ops_list_head == NULL) { - win_ptr->num_targets_with_pending_net_ops--; - MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); - if (win_ptr->num_targets_with_pending_net_ops == 0) { - MPIDI_CH3I_Win_set_inactive(win_ptr); - } - } - - fn_exit: - MPIDI_RMA_FUNC_EXIT(MPID_STATE_CHECK_AND_SET_REQ_COMPLETION); - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -#undef FUNCNAME #define FUNCNAME issue_ops_target #undef FCNAME #define FCNAME MPIU_QUOTE(FUNCNAME) http://git.mpich.org/mpich.git/commitdiff/16da069598460e9482bbce66ed108c0eb… commit 16da069598460e9482bbce66ed108c0ebed980f9 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Sun Aug 2 21:48:35 2015 -0500 Split pending op list into pending_net_ops_list and pending_user_ops_list. Here we split pending op list in the target object into two lists: (1) pending_net_ops_list and (2) pending_user_ops_list. (1) is used to store operations that are held to wait for network events (synchronization), and (2) is used to store operations that are held to wait for user events (MPI_Win_unlock/flush/flush_local posted). When user events happen, we move the operation in (2) to (1). Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpid_rma_oplist.h b/src/mpid/ch3/include/mpid_rma_oplist.h index bf2cd7d..1702f45 100644 --- a/src/mpid/ch3/include/mpid_rma_oplist.h +++ b/src/mpid/ch3/include/mpid_rma_oplist.h @@ -54,7 +54,8 @@ MPIR_T_PVAR_DOUBLE_TIMER_DECL_EXTERN(RMA, rma_rmaqueue_alloc); (win_)->states.access_state != MPIDI_RMA_LOCK_ALL_ISSUED && \ (target_)->access_state != MPIDI_RMA_LOCK_CALLED && \ (target_)->access_state != MPIDI_RMA_LOCK_ISSUED && \ - (target_)->pending_op_list_head == NULL && \ + (target_)->pending_net_ops_list_head == NULL && \ + (target_)->pending_user_ops_list_head == NULL && \ (target_)->issued_read_op_list_head == NULL && \ (target_)->issued_write_op_list_head == NULL && \ (target_)->issued_dt_op_list_head == NULL) { \ @@ -300,7 +301,8 @@ static inline MPIDI_RMA_Target_t *MPIDI_CH3I_Win_target_alloc(MPID_Win * win_ptr e->issued_read_op_list_head = NULL; e->issued_write_op_list_head = NULL; e->issued_dt_op_list_head = NULL; - e->pending_op_list_head = NULL; + e->pending_net_ops_list_head = NULL; + e->pending_user_ops_list_head = NULL; e->next_op_to_issue = NULL; e->target_rank = -1; @@ -333,7 +335,8 @@ static inline int MPIDI_CH3I_Win_target_free(MPID_Win * win_ptr, MPIDI_RMA_Targe MPIU_Assert(e->issued_read_op_list_head == NULL); MPIU_Assert(e->issued_write_op_list_head == NULL); MPIU_Assert(e->issued_dt_op_list_head == NULL); - MPIU_Assert(e->pending_op_list_head == NULL); + MPIU_Assert(e->pending_net_ops_list_head == NULL); + MPIU_Assert(e->pending_user_ops_list_head == NULL); /* use PREPEND when return objects back to the pool * in order to improve cache performance */ @@ -451,18 +454,39 @@ static inline int MPIDI_CH3I_Win_enqueue_op(MPID_Win * win_ptr, MPIDI_RMA_Op_t * } } - if (target->pending_op_list_head == NULL) - win_ptr->num_targets_with_pending_net_ops++; + /* Note that if it is a request-based RMA, do not put it in pending user list, + * otherwise a wait call before unlock will be blocked. */ + if (MPIR_CVAR_CH3_RMA_DELAY_ISSUING_FOR_PIGGYBACKING && op->ureq == NULL) { + if (target->pending_user_ops_list_head != NULL) { + MPIDI_RMA_Op_t *user_op = target->pending_user_ops_list_head; + /* Move head element of user pending list to net pending list */ + if (target->pending_net_ops_list_head == NULL) + win_ptr->num_targets_with_pending_net_ops++; + MPL_DL_DELETE(target->pending_user_ops_list_head, user_op); + MPL_DL_APPEND(target->pending_net_ops_list_head, user_op); + + if (target->next_op_to_issue == NULL) + target->next_op_to_issue = user_op; + } - /* Enqueue operation into pending list. */ - MPL_DL_APPEND(target->pending_op_list_head, op); - if (target->next_op_to_issue == NULL) - target->next_op_to_issue = op; + /* Enqueue operation into user pending list. */ + MPL_DL_APPEND(target->pending_user_ops_list_head, op); + } + else { + /* Enqueue operation into net pending list. */ + if (target->pending_net_ops_list_head == NULL) + win_ptr->num_targets_with_pending_net_ops++; + MPL_DL_APPEND(target->pending_net_ops_list_head, op); + + if (target->next_op_to_issue == NULL) + target->next_op_to_issue = op; + } - if (win_ptr->states.access_state == MPIDI_RMA_FENCE_GRANTED || - win_ptr->states.access_state == MPIDI_RMA_PSCW_GRANTED || - win_ptr->states.access_state == MPIDI_RMA_LOCK_ALL_GRANTED || - target->access_state == MPIDI_RMA_LOCK_GRANTED) + if (target->pending_net_ops_list_head != NULL && + (win_ptr->states.access_state == MPIDI_RMA_FENCE_GRANTED || + win_ptr->states.access_state == MPIDI_RMA_PSCW_GRANTED || + win_ptr->states.access_state == MPIDI_RMA_LOCK_ALL_GRANTED || + target->access_state == MPIDI_RMA_LOCK_GRANTED)) MPIDI_CH3I_Win_set_active(win_ptr); fn_exit: diff --git a/src/mpid/ch3/include/mpid_rma_types.h b/src/mpid/ch3/include/mpid_rma_types.h index 2763af5..3014c0a 100644 --- a/src/mpid/ch3/include/mpid_rma_types.h +++ b/src/mpid/ch3/include/mpid_rma_types.h @@ -68,7 +68,8 @@ typedef struct MPIDI_RMA_Target { struct MPIDI_RMA_Op *issued_read_op_list_head; struct MPIDI_RMA_Op *issued_write_op_list_head; struct MPIDI_RMA_Op *issued_dt_op_list_head; - struct MPIDI_RMA_Op *pending_op_list_head; + struct MPIDI_RMA_Op *pending_net_ops_list_head; /* pending operations that are waiting for network events */ + struct MPIDI_RMA_Op *pending_user_ops_list_head; /* pending operations that are waiting for user events */ struct MPIDI_RMA_Op *next_op_to_issue; struct MPIDI_RMA_Target *next; struct MPIDI_RMA_Target *prev; diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index 7b3868e..91816f0 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -616,13 +616,13 @@ static inline int handle_lock_ack(MPID_Win * win_ptr, int target_rank, MPIDI_CH3 if (flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_GRANTED) { t->access_state = MPIDI_RMA_LOCK_GRANTED; - if (t->pending_op_list_head) + if (t->pending_net_ops_list_head) MPIDI_CH3I_Win_set_active(win_ptr); } if (win_ptr->states.access_state == MPIDI_RMA_LOCK_ALL_GRANTED || t->access_state == MPIDI_RMA_LOCK_GRANTED) { - if (t->pending_op_list_head == NULL) { + if (t->pending_net_ops_list_head == NULL) { int made_progress ATTRIBUTE((unused)) = 0; mpi_errno = MPIDI_CH3I_RMA_Make_progress_target(win_ptr, t->target_rank, &made_progress); @@ -704,9 +704,9 @@ static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, } } - MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_op_list_head), op); + MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_net_ops_list_head), op); - if (target->pending_op_list_head == NULL) { + if (target->pending_net_ops_list_head == NULL) { win_ptr->num_targets_with_pending_net_ops--; MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); if (win_ptr->num_targets_with_pending_net_ops == 0) { diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 7eb23cd..54c5bc5 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -105,13 +105,34 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta if (target == NULL) goto fn_exit; + /* When user event happens, move op in user pending list to network pending list */ + if (target->sync.sync_flag == MPIDI_RMA_SYNC_FLUSH || + target->sync.sync_flag == MPIDI_RMA_SYNC_FLUSH_LOCAL || + target->sync.sync_flag == MPIDI_RMA_SYNC_UNLOCK || target->win_complete_flag) { + + MPIDI_RMA_Op_t *user_op = target->pending_user_ops_list_head; + + if (user_op != NULL) { + if (target->pending_net_ops_list_head == NULL) + win_ptr->num_targets_with_pending_net_ops++; + + MPL_DL_DELETE(target->pending_user_ops_list_head, user_op); + MPL_DL_APPEND(target->pending_net_ops_list_head, user_op); + + if (target->next_op_to_issue == NULL) + target->next_op_to_issue = user_op; + } + } + switch (target->access_state) { case MPIDI_RMA_LOCK_CALLED: if (target->sync.sync_flag == MPIDI_RMA_SYNC_NONE || target->sync.sync_flag == MPIDI_RMA_SYNC_FLUSH_LOCAL || target->sync.sync_flag == MPIDI_RMA_SYNC_FLUSH) { - if (target->pending_op_list_head == NULL || - !target->pending_op_list_head->piggyback_lock_candidate) { + if ((target->pending_net_ops_list_head == NULL || + !target->pending_net_ops_list_head->piggyback_lock_candidate) && + (target->pending_user_ops_list_head == NULL || + !target->pending_user_ops_list_head->piggyback_lock_candidate)) { /* issue lock request */ target->access_state = MPIDI_RMA_LOCK_ISSUED; if (target->target_rank == rank) { @@ -129,7 +150,7 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta } } else if (target->sync.sync_flag == MPIDI_RMA_SYNC_UNLOCK) { - if (target->pending_op_list_head == NULL) { + if (target->pending_net_ops_list_head == NULL) { /* No RMA operation has ever been posted to this target, * finish issuing, no need to acquire the lock. Cleanup * function will clean it up. */ @@ -145,8 +166,8 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta * in pending list, this operation must be the only operation * and it is prepared to piggyback LOCK and UNLOCK. */ MPIU_Assert(MPIR_CVAR_CH3_RMA_DELAY_ISSUING_FOR_PIGGYBACKING); - MPIU_Assert(target->pending_op_list_head->next == NULL); - MPIU_Assert(target->pending_op_list_head->piggyback_lock_candidate); + MPIU_Assert(target->pending_net_ops_list_head->next == NULL); + MPIU_Assert(target->pending_net_ops_list_head->piggyback_lock_candidate); } } break; @@ -154,7 +175,7 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta case MPIDI_RMA_LOCK_GRANTED: case MPIDI_RMA_NONE: if (target->win_complete_flag) { - if (target->pending_op_list_head == NULL) { + if (target->pending_net_ops_list_head == NULL) { MPIDI_CH3_Pkt_flags_t flags = MPIDI_CH3_PKT_FLAG_NONE; if (target->sync.sync_flag == MPIDI_RMA_SYNC_FLUSH && target->put_acc_issued) { flags |= MPIDI_CH3_PKT_FLAG_RMA_FLUSH; @@ -172,7 +193,7 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta } } else if (target->sync.sync_flag == MPIDI_RMA_SYNC_FLUSH) { - if (target->pending_op_list_head == NULL) { + if (target->pending_net_ops_list_head == NULL) { if (target->target_rank != rank) { if (target->put_acc_issued) { @@ -191,7 +212,7 @@ static inline int check_and_switch_target_state(MPID_Win * win_ptr, MPIDI_RMA_Ta } } else if (target->sync.sync_flag == MPIDI_RMA_SYNC_UNLOCK) { - if (target->pending_op_list_head == NULL) { + if (target->pending_net_ops_list_head == NULL) { if (target->target_rank == rank) { mpi_errno = MPIDI_CH3I_Release_lock(win_ptr); if (mpi_errno != MPI_SUCCESS) @@ -289,7 +310,7 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar MPIU_ERR_POP(mpi_errno); } } - MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_op_list_head), rma_op); + MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_net_ops_list_head), rma_op); (*op_completed) = TRUE; } @@ -301,13 +322,13 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar MPIU_ERR_POP(mpi_errno); } - MPIDI_CH3I_RMA_Ops_unlink(&(target->pending_op_list_head), rma_op); + MPIDI_CH3I_RMA_Ops_unlink(&(target->pending_net_ops_list_head), rma_op); MPIDI_CH3I_RMA_Ops_append(list_ptr, rma_op); win_ptr->active_req_cnt += rma_op->ref_cnt; } - if (target->pending_op_list_head == NULL) { + if (target->pending_net_ops_list_head == NULL) { win_ptr->num_targets_with_pending_net_ops--; MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); if (win_ptr->num_targets_with_pending_net_ops == 0) { @@ -337,7 +358,7 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ (*made_progress) = 0; if (win_ptr->num_targets_with_pending_net_ops == 0 || target == NULL || - target->pending_op_list_head == NULL) + target->pending_net_ops_list_head == NULL) goto fn_exit; /* Issue out operations in the list. */ @@ -352,18 +373,6 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ break; } - if (MPIR_CVAR_CH3_RMA_DELAY_ISSUING_FOR_PIGGYBACKING && curr_op->next == NULL && - target->sync.sync_flag == MPIDI_RMA_SYNC_NONE && curr_op->ureq == NULL) { - /* If DELAY_ISSUING_FOR_PIGGYBACKING is turned on, - * skip the last OP if sync_flag is NONE since we - * want to leave it to the ending synchronization - * so that we can piggyback UNLOCK / FLUSH. - * However, if it is a request-based RMA, do not - * skip it (otherwise a wait call before unlock - * will be blocked). */ - break; - } - flags = MPIDI_CH3_PKT_FLAG_NONE; if (first_op) { @@ -630,7 +639,8 @@ int MPIDI_CH3I_RMA_Cleanup_ops_aggressive(MPID_Win * win_ptr) for (i = 0; i < win_ptr->num_slots; i++) { if (win_ptr->slots[i].target_list_head != NULL) { curr_target = win_ptr->slots[i].target_list_head; - while (curr_target != NULL && curr_target->pending_op_list_head == NULL) + while (curr_target != NULL && curr_target->pending_net_ops_list_head == NULL && + curr_target->pending_user_ops_list_head == NULL) curr_target = curr_target->next; if (curr_target != NULL) break; http://git.mpich.org/mpich.git/commitdiff/277290f386a2c08f8ff08f3c28e376dbe… commit 277290f386a2c08f8ff08f3c28e376dbecb84579 Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 09:51:50 2015 -0500 Rename variable num_targets_with_pending_ops to num_targets_with_pending_net_ops Originally we use num_targets_with_pending_ops to keep track of number of targets that have non-empty pending operation list. Now we want to only keep track of targets that have non-empty pending operation list for network events. We rename the variable here. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpid_rma_oplist.h b/src/mpid/ch3/include/mpid_rma_oplist.h index 62637af..bf2cd7d 100644 --- a/src/mpid/ch3/include/mpid_rma_oplist.h +++ b/src/mpid/ch3/include/mpid_rma_oplist.h @@ -452,7 +452,7 @@ static inline int MPIDI_CH3I_Win_enqueue_op(MPID_Win * win_ptr, MPIDI_RMA_Op_t * } if (target->pending_op_list_head == NULL) - win_ptr->num_targets_with_pending_ops++; + win_ptr->num_targets_with_pending_net_ops++; /* Enqueue operation into pending list. */ MPL_DL_APPEND(target->pending_op_list_head, op); diff --git a/src/mpid/ch3/include/mpidpre.h b/src/mpid/ch3/include/mpidpre.h index bc61055..6390bbc 100644 --- a/src/mpid/ch3/include/mpidpre.h +++ b/src/mpid/ch3/include/mpidpre.h @@ -329,8 +329,9 @@ typedef struct MPIDI_Win_basic_info { enum MPIDI_RMA_states access_state; \ enum MPIDI_RMA_states exposure_state; \ } states; \ - int num_targets_with_pending_ops; /* keep track of number of targets \ - tht has pending operations. */ \ + int num_targets_with_pending_net_ops; /* keep track of number of \ + targets that has non-empty \ + net pending op list. */ \ int active_req_cnt; /* keep track of number of active requests in \ current epoch, i.e., number of issued but \ incomplete RMA operations. */ \ diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index d47258c..7b3868e 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -592,7 +592,7 @@ static inline int handle_lock_ack(MPID_Win * win_ptr, int target_rank, MPIDI_CH3 if (win_ptr->outstanding_locks == 0) { win_ptr->states.access_state = MPIDI_RMA_LOCK_ALL_GRANTED; - if (win_ptr->num_targets_with_pending_ops) { + if (win_ptr->num_targets_with_pending_net_ops) { mpi_errno = MPIDI_CH3I_Win_set_active(win_ptr); if (mpi_errno != MPI_SUCCESS) { MPIU_ERR_POP(mpi_errno); @@ -707,9 +707,9 @@ static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, MPIDI_CH3I_RMA_Ops_free_elem(win_ptr, &(target->pending_op_list_head), op); if (target->pending_op_list_head == NULL) { - win_ptr->num_targets_with_pending_ops--; - MPIU_Assert(win_ptr->num_targets_with_pending_ops >= 0); - if (win_ptr->num_targets_with_pending_ops == 0) { + win_ptr->num_targets_with_pending_net_ops--; + MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); + if (win_ptr->num_targets_with_pending_net_ops == 0) { MPIDI_CH3I_Win_set_inactive(win_ptr); } } diff --git a/src/mpid/ch3/src/ch3u_rma_progress.c b/src/mpid/ch3/src/ch3u_rma_progress.c index 82d0df4..7eb23cd 100644 --- a/src/mpid/ch3/src/ch3u_rma_progress.c +++ b/src/mpid/ch3/src/ch3u_rma_progress.c @@ -308,9 +308,9 @@ static inline int check_and_set_req_completion(MPID_Win * win_ptr, MPIDI_RMA_Tar } if (target->pending_op_list_head == NULL) { - win_ptr->num_targets_with_pending_ops--; - MPIU_Assert(win_ptr->num_targets_with_pending_ops >= 0); - if (win_ptr->num_targets_with_pending_ops == 0) { + win_ptr->num_targets_with_pending_net_ops--; + MPIU_Assert(win_ptr->num_targets_with_pending_net_ops >= 0); + if (win_ptr->num_targets_with_pending_net_ops == 0) { MPIDI_CH3I_Win_set_inactive(win_ptr); } } @@ -336,7 +336,7 @@ static inline int issue_ops_target(MPID_Win * win_ptr, MPIDI_RMA_Target_t * targ (*made_progress) = 0; - if (win_ptr->num_targets_with_pending_ops == 0 || target == NULL || + if (win_ptr->num_targets_with_pending_net_ops == 0 || target == NULL || target->pending_op_list_head == NULL) goto fn_exit; diff --git a/src/mpid/ch3/src/ch3u_rma_sync.c b/src/mpid/ch3/src/ch3u_rma_sync.c index 9b2c726..92fbccd 100644 --- a/src/mpid/ch3/src/ch3u_rma_sync.c +++ b/src/mpid/ch3/src/ch3u_rma_sync.c @@ -458,7 +458,7 @@ static int fence_barrier_complete(MPID_Request * sreq) if (win_ptr->states.access_state == MPIDI_RMA_FENCE_ISSUED) { win_ptr->states.access_state = MPIDI_RMA_FENCE_GRANTED; - if (win_ptr->num_targets_with_pending_ops) { + if (win_ptr->num_targets_with_pending_net_ops) { mpi_errno = MPIDI_CH3I_Win_set_active(win_ptr); if (mpi_errno != MPI_SUCCESS) { MPIU_ERR_POP(mpi_errno); @@ -823,7 +823,7 @@ static int start_req_complete(MPID_Request * req) if (win_ptr->sync_request_cnt == 0) { win_ptr->states.access_state = MPIDI_RMA_PSCW_GRANTED; - if (win_ptr->num_targets_with_pending_ops) + if (win_ptr->num_targets_with_pending_net_ops) MPIDI_CH3I_Win_set_active(win_ptr); } diff --git a/src/mpid/ch3/src/mpid_rma.c b/src/mpid/ch3/src/mpid_rma.c index cba9e8e..025fbba 100644 --- a/src/mpid/ch3/src/mpid_rma.c +++ b/src/mpid/ch3/src/mpid_rma.c @@ -288,7 +288,7 @@ static int win_init(MPI_Aint size, int disp_unit, int create_flavor, int model, (*win_ptr)->shm_allocated = FALSE; (*win_ptr)->states.access_state = MPIDI_RMA_NONE; (*win_ptr)->states.exposure_state = MPIDI_RMA_NONE; - (*win_ptr)->num_targets_with_pending_ops = 0; + (*win_ptr)->num_targets_with_pending_net_ops = 0; (*win_ptr)->active_req_cnt = 0; (*win_ptr)->start_ranks_in_win_grp = NULL; (*win_ptr)->start_grp_size = 0; http://git.mpich.org/mpich.git/commitdiff/e14a94eed62a1db783020023aed712c55… commit e14a94eed62a1db783020023aed712c558620cdb Author: Xin Zhao <xinzhao3(a)illinois.edu> Date: Mon Aug 3 13:31:48 2015 -0500 Bug-fix when Lock Ack is LOCK_QUEUED_DATA_QUEUED. When flag in Lock Ack is LOCK_QUEUED_DATA_QUEUED, it means that the entire operation is queued on the target side, at this point, we should not complete and delete that operation. When origin side receives Lock Ack with LOCK_GRANTED, it will delete that operation from the list. Signed-off-by: Pavan Balaji <balaji(a)anl.gov> diff --git a/src/mpid/ch3/include/mpidrma.h b/src/mpid/ch3/include/mpidrma.h index c0f843a..d47258c 100644 --- a/src/mpid/ch3/include/mpidrma.h +++ b/src/mpid/ch3/include/mpidrma.h @@ -663,8 +663,7 @@ static inline int handle_lock_ack_with_op(MPID_Win * win_ptr, MPIU_Assert(op_flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_SHARED || op_flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_EXCLUSIVE); - if (flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_GRANTED || - flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_QUEUED_DATA_QUEUED) { + if (flags & MPIDI_CH3_PKT_FLAG_RMA_LOCK_GRANTED) { if (op->reqs_size > 0) { MPID_Request **req = NULL; ----------------------------------------------------------------------- Summary of changes: src/mpid/ch3/include/mpid_rma_oplist.h | 50 ++++++++--- src/mpid/ch3/include/mpid_rma_types.h | 3 +- src/mpid/ch3/include/mpidpre.h | 5 +- src/mpid/ch3/include/mpidrma.h | 127 +++++++++++++++++++++-------- src/mpid/ch3/src/ch3u_rma_progress.c | 142 ++++++++------------------------ src/mpid/ch3/src/ch3u_rma_sync.c | 4 +- src/mpid/ch3/src/mpid_rma.c | 2 +- 7 files changed, 171 insertions(+), 162 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-56-gd0372e0
by noreply＠mpich.org 03 Aug '15

03 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via d0372e05410f30a9ea260609325417ec58479905 (commit) from 219757b2727675abed69dd188fd6bd8a27133e21 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/d0372e05410f30a9ea260609325417ec5… commit d0372e05410f30a9ea260609325417ec58479905 Author: Pavan Balaji <balaji(a)anl.gov> Date: Mon Aug 3 13:58:43 2015 -0500 Added more mailmaps to deal with changed email addresses. No reviewer. diff --git a/.mailmap b/.mailmap index 98e6805..f37f76c 100644 --- a/.mailmap +++ b/.mailmap @@ -4,5 +4,41 @@ # # See "man git-shortlog" for formatting information on this file. -# use "Dave Goodell" as Dave's canonical name (SVN history includes "David"). +Andrey Lobanov <andrey.lobanov(a)intel.com> + +Antonio J. Pena <apenya(a)mcs.anl.gov> + +Artem V. Yalozo <artem.v.yalozo(a)intel.com> + +Charles J Archer <charles.j.archer(a)intel.com> <archerc(a)us.ibm.com> + Dave Goodell <goodell(a)mcs.anl.gov> + +Halim Amer <aamer(a)anl.gov> <halim.amer(a)gmail.com> + +James Dinan <james.dinan(a)intel.com> <dinan(a)mcs.anl.gov> + +Jeff Hammond <jeff.r.hammond(a)intel.com> <jeff.science(a)gmail.com> +Jeff Hammond <jeff.r.hammond(a)intel.com> <jhammond(a)alcf.anl.gov> + +Junchao Zhang <jczhang(a)mcs.anl.gov> <jczhang(a)bblogin2.mcs.anl.gov> + +Keira Zhang <keirazhang(a)cn.ibm.com> + +Masamichi Takagi <masamichi.takagi(a)riken.jp> <masamichi.takagi(a)gmail.com> + +Norio Yamaguchi <norio.yamaguchi(a)riken.jp> <yamaguchi(a)kangaroo2.aics27.riken.jp> +Norio Yamaguchi <norio.yamaguchi(a)riken.jp> + +Paul Coffman <pkcoff(a)alcf.anl.gov> <pkcoff(a)vestalac1.ftd.alcf.anl.gov> +Paul Coffman <pkcoff(a)alcf.anl.gov> <pkcoff(a)us.ibm.com> + +Pavan Balaji <balaji(a)anl.gov> <balaji(a)mcs.anl.gov> +Pavan Balaji <balaji(a)anl.gov> <balaji(a)bb95.mcs.anl.gov> + +Sameh Sharkawi <sssharka(a)us.ibm.com> + +Wesley Bland <wesley.bland(a)intel.com> <wbland(a)anl.gov> +Wesley Bland <wesley.bland(a)intel.com> <wbland(a)mcs.anl.gov> + +William Gropp <wgropp(a)illinois.edu> <gropp(a)mcs.anl.gov> ----------------------------------------------------------------------- Summary of changes: .mailmap | 38 +++++++++++++++++++++++++++++++++++++- 1 files changed, 37 insertions(+), 1 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-55-g219757b
by noreply＠mpich.org 03 Aug '15

03 Aug '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 219757b2727675abed69dd188fd6bd8a27133e21 (commit) from 2672616c69f4e131c830744c3e53d07f1986cb92 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/219757b2727675abed69dd188fd6bd8a2… commit 219757b2727675abed69dd188fd6bd8a27133e21 Author: Pavan Balaji <balaji(a)anl.gov> Date: Sat Aug 1 11:36:30 2015 -0500 Update author list for the MPICH user guides. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/doc/installguide/install.tex.vin b/doc/installguide/install.tex.vin index b256706..3b21843 100644 --- a/doc/installguide/install.tex.vin +++ b/doc/installguide/install.tex.vin @@ -49,9 +49,10 @@ Argonne National Laboratory} \author{ -Pavan Balaji \and Wesley Bland \and William Gropp \and Rob Latham \and -Huiwei Lu \and Antonio J. Pe\~na \and Ken Raffenetti \and Sangmin Seo -\and Rajeev Thakur \and Junchao Zhang +Abdelhalim Amer \and Pavan Balaji \and Wesley Bland \and William Gropp +\and Rob Latham \and Huiwei Lu \and Lena Oden \and Antonio J. Pe\~na +\and Ken Raffenetti \and Sangmin Seo \and Rajeev Thakur \and Junchao +Zhang } \maketitle diff --git a/doc/userguide/user.tex.vin b/doc/userguide/user.tex.vin index d4ae978..d255fd2 100644 --- a/doc/userguide/user.tex.vin +++ b/doc/userguide/user.tex.vin @@ -47,9 +47,10 @@ Mathematics and Computer Science Division\\ Argonne National Laboratory} \author{ -Pavan Balaji \and Wesley Bland \and William Gropp \and Rob Latham \and -Huiwei Lu \and Antonio J. Pe\~na \and Ken Raffenetti \and Sangmin Seo -\and Rajeev Thakur \and Junchao Zhang +Abdelhalim Amer \and Pavan Balaji \and Wesley Bland \and William Gropp +\and Rob Latham \and Huiwei Lu \and Lena Oden \and Antonio J. Pe\~na +\and Ken Raffenetti \and Sangmin Seo \and Rajeev Thakur \and Junchao +Zhang } \maketitle ----------------------------------------------------------------------- Summary of changes: doc/installguide/install.tex.vin | 7 ++++--- doc/userguide/user.tex.vin | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) hooks/post-receive -- MPICH primary repository

1 0