November 2013 - Triton-commits - mailman3.cels.anl.gov

branch, master, updated. c7fd5d2831bfc8c1961c660da333ea4c8eef6fdf
by noreply＠mcs.anl.gov 27 Nov '13

27 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via c7fd5d2831bfc8c1961c660da333ea4c8eef6fdf (commit) from 154b13017653474b4fa28812a84a135769d3b25e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit c7fd5d2831bfc8c1961c660da333ea4c8eef6fdf Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 27 14:51:48 2013 -0500 presentation for december asg concall ----------------------------------------------------------------------- Summary of changes: .../anl-asg-concall-presentation-2013-12-02.pdf | Bin 0 -> 2846284 bytes .../anl-asg-concall-presentation-2013-12-02.pptx | Bin 0 -> 2237578 bytes .../asg-concall-2013-12-02/blockdiagram.png | Bin 0 -> 26464 bytes presentations/asg-concall-2013-12-02/demo.txt | 52 ++++++++++++++++++++ .../asg-concall-2013-12-02/mercury-na.png | Bin 0 -> 17066 bytes .../asg-concall-2013-12-02/screenshot-demo.png | Bin 0 -> 91572 bytes presentations/asg-concall-2013-12-02/tableview.png | Bin 0 -> 1926515 bytes 7 files changed, 52 insertions(+), 0 deletions(-) create mode 100644 presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pdf create mode 100755 presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pptx create mode 100644 presentations/asg-concall-2013-12-02/blockdiagram.png create mode 100644 presentations/asg-concall-2013-12-02/demo.txt create mode 100644 presentations/asg-concall-2013-12-02/mercury-na.png create mode 100644 presentations/asg-concall-2013-12-02/screenshot-demo.png create mode 100644 presentations/asg-concall-2013-12-02/tableview.png Diff of changes: diff --git a/presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pdf b/presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pdf new file mode 100644 index 0000000..15811d2 Binary files /dev/null and b/presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pdf differ diff --git a/presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pptx b/presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pptx new file mode 100755 index 0000000..47c8752 Binary files /dev/null and b/presentations/asg-concall-2013-12-02/anl-asg-concall-presentation-2013-12-02.pptx differ diff --git a/presentations/asg-concall-2013-12-02/blockdiagram.png b/presentations/asg-concall-2013-12-02/blockdiagram.png new file mode 100644 index 0000000..69cd869 Binary files /dev/null and b/presentations/asg-concall-2013-12-02/blockdiagram.png differ diff --git a/presentations/asg-concall-2013-12-02/demo.txt b/presentations/asg-concall-2013-12-02/demo.txt new file mode 100644 index 0000000..e80434e --- /dev/null +++ b/presentations/asg-concall-2013-12-02/demo.txt @@ -0,0 +1,52 @@ +$ src/server/triton-server --daemonize --hostfile 4.txt --listenaddr tcp://localhost:3344 +$ src/server/triton-server --daemonize --hostfile 4.txt --listenaddr tcp://localhost:3345 +$ src/server/triton-server --daemonize --hostfile 4.txt --listenaddr tcp://localhost:3346 +$ src/server/triton-server --daemonize --hostfile 4.txt --listenaddr tcp://localhost:3347 + +$ src/admin-tools/triton-show-system-state tcp://localhost:3344 <node> +<key> <value> +================================================= + ::3344 addr tcp://localhost:3344 + ::3345 addr tcp://localhost:3345 + ::3346 addr tcp://localhost:3346 + ::3347 addr tcp://localhost:3347 + ::3344 status up + ::3345 status unknown + ::3346 status unknown + ::3347 status unknown + +$ src/admin-tools/triton-show-system-state tcp://localhost:3344 + <node> <key> <value> +================================================= + ::3344 addr tcp://localhost:3344 + ::3345 addr tcp://localhost:3345 + ::3346 addr tcp://localhost:3346 + ::3347 addr tcp://localhost:3347 + ::3344 status up + ::3345 status up + ::3346 status up + ::3347 status up + +$ src/admin-tools/triton-ls tcp://localhost:3344 +Objects on server ::3344 (tcp://localhost:3344): +Objects on server ::3345 (tcp://localhost:3345): +Objects on server ::3346 (tcp://localhost:3346): +Objects on server ::3347 (tcp://localhost:3347): + +$ src/admin-tools/triton-cp tcp://localhost:3344 posix:/etc/hosts ::4000 3 + +$ src/admin-tools/triton-ls tcp://localhost:3344 +Objects on server ::3344 (tcp://localhost:3344): +Objects on server ::3345 (tcp://localhost:3345): + ::4000 +Objects on server ::3346 (tcp://localhost:3346): + ::4000 +Objects on server ::3347 (tcp://localhost:3347): + ::4000 + +$ src/admin-tools/triton-shutdown-all-servers tcp://localhost:3344 + +$ src/admin-tools/triton-ls tcp://localhost:3344 +triton_attach(): :::remote_triton_rpc_system_state_get_snapshot(), +base:Mercury Error + diff --git a/presentations/asg-concall-2013-12-02/mercury-na.png b/presentations/asg-concall-2013-12-02/mercury-na.png new file mode 100644 index 0000000..df2b5b3 Binary files /dev/null and b/presentations/asg-concall-2013-12-02/mercury-na.png differ diff --git a/presentations/asg-concall-2013-12-02/screenshot-demo.png b/presentations/asg-concall-2013-12-02/screenshot-demo.png new file mode 100644 index 0000000..40f38a1 Binary files /dev/null and b/presentations/asg-concall-2013-12-02/screenshot-demo.png differ diff --git a/presentations/asg-concall-2013-12-02/tableview.png b/presentations/asg-concall-2013-12-02/tableview.png new file mode 100644 index 0000000..b1c57f4 Binary files /dev/null and b/presentations/asg-concall-2013-12-02/tableview.png differ hooks/post-receive --

1 0

branch, master, updated. 128a736ad62a003759027ce7fdfedefb55f6d77f
by noreply＠mcs.anl.gov 25 Nov '13

25 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 128a736ad62a003759027ce7fdfedefb55f6d77f (commit) via c421a9fdfbb21543740d39c1b080fbcc3e2a9114 (commit) via 6be7b1c5520c230b612b750310143918e91cbc31 (commit) from 591fb2a9b70847edc5e959bd609261a918847e52 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 128a736ad62a003759027ce7fdfedefb55f6d77f Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Mon Nov 25 14:42:28 2013 -0600 update rosd routines for new triton_uint128_to_string method commit c421a9fdfbb21543740d39c1b080fbcc3e2a9114 Merge: 6be7b1c5520c230b612b750310143918e91cbc31 591fb2a9b70847edc5e959bd609261a918847e52 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Mon Nov 25 13:44:15 2013 -0600 Merge branch 'master' of git.mcs.anl.gov:triton commit 6be7b1c5520c230b612b750310143918e91cbc31 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Mon Nov 25 13:43:59 2013 -0600 ipv6 formatting for oids in strings ----------------------------------------------------------------------- Summary of changes: code/scripts/launch_servers.sh | 2 +- code/src/admin-tools/triton-ls.ae | 6 ++-- code/src/admin-tools/triton-rm.ae | 32 +++++++------------ code/src/common/node.c | 36 ----------------------- code/src/common/node.h | 5 +-- code/src/replicated-osd/rosd-create.ae | 13 +++++--- code/src/replicated-osd/rosd-remove.ae | 18 +++++++---- code/src/replicated-osd/rosd-txn-nr-cache.ae | 19 ++++++------ code/src/replicated-osd/rosd-write.ae | 8 +++-- code/src/replicated-osd/rosd.ae | 5 ++- code/src/system-state/system-state.ae | 6 ++-- code/src/transactional-osd/tosd-fd-cache.ae | 24 ++++++++------- code/src/transactional-osd/transactional-osd.ae | 8 +++-- code/tests/placement/test-placement.c | 29 ++++++++++++------ code/tests/system-state/test-system-state.ae | 2 +- code/tests/test-util.sh | 2 +- code/tests/triton-rm.sh | 10 +++--- code/tests/triton-start-test-servers.sh | 2 +- code/tests/triton-touch.sh | 2 +- 19 files changed, 104 insertions(+), 125 deletions(-) Diff of changes: diff --git a/code/scripts/launch_servers.sh b/code/scripts/launch_servers.sh index 65a484e..553fe93 100755 --- a/code/scripts/launch_servers.sh +++ b/code/scripts/launch_servers.sh @@ -56,7 +56,7 @@ echo -n > $HOSTFILE endport=$(($g_baseport + $g_nservers - 1)) for p in $(seq $g_baseport $endport) do - echo -e "$p.0\t${LADDR}:$p" >> $HOSTFILE + echo -e "$p::0\t${LADDR}:$p" >> $HOSTFILE done # create screen commands diff --git a/code/src/admin-tools/triton-ls.ae b/code/src/admin-tools/triton-ls.ae index df69ea4..4e2273d 100644 --- a/code/src/admin-tools/triton-ls.ae +++ b/code/src/admin-tools/triton-ls.ae @@ -30,7 +30,7 @@ __blocking int aesop_main(int argc, char **argv) uint128_t upper; uint128_t *oid_array; uint32_t oid_array_count; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; if(argc != 2) { @@ -126,9 +126,9 @@ __blocking int aesop_main(int argc, char **argv) &next_position); for(j=0; !triton_is_error(tret) && j<oid_array_count; j++) { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, - oid_array[j]); + oid_str = triton_uint128_to_string(oid_array[j]); printf("\t%s\n", oid_str); + if (oid_str) free(oid_str); } lower = next_position; } diff --git a/code/src/admin-tools/triton-rm.ae b/code/src/admin-tools/triton-rm.ae index d65ca92..2883eb9 100644 --- a/code/src/admin-tools/triton-rm.ae +++ b/code/src/admin-tools/triton-rm.ae @@ -91,23 +91,14 @@ void parse_args (int *argc, for (i=optind; i < *argc; i++) { - uint64_t u; - uint64_t l; - int n; - - n = sscanf((*argv)[i], "%" SCNu64 ".%" SCNu64, &u, &l); - if (n == 2) - { - (*oids)[*oid_count].u = u; - (*oids)[*oid_count].l = l; - *oid_count += 1; - } - else - { - if (*verbose) { - fprintf(stderr, "invalid oid: %s\n", (*argv)[i]); - } - } + uint128_t oid; + + oid = triton_uint128_from_string((*argv)[i]); + triton_uint128_set(oid, (*oids)[*oid_count]); + *oid_count += 1; + //if (*verbose) { + // fprintf(stderr, "invalid oid: %s\n", (*argv)[i]); + //} } } @@ -173,9 +164,10 @@ __blocking int aesop_main(int argc, char **argv) { if (verbose) { - char oidstr[TRITON_UINT128_STRLEN]; - triton_uint128_to_string(oidstr, sizeof(oidstr), oids[i]); - printf("removing: %s\n", oidstr); + char *oid_str; + oid_str = triton_uint128_to_string(oids[i]); + printf("removing: %s\n", oid_str); + if (oid_str) free(oid_str); } tret = remote_triton_rpc_rosd_remove (oids[i], ROSD_FLAG_CHAIN); if (triton_is_error(tret)) diff --git a/code/src/common/node.c b/code/src/common/node.c index b757613..b71aaf3 100644 --- a/code/src/common/node.c +++ b/code/src/common/node.c @@ -25,42 +25,6 @@ int triton_cmp_node(triton_node_t a, triton_node_t b) return 0; } -char* triton_node_to_string(triton_node_t node) -{ - char* tmp = malloc(34); - if(!tmp) - { - return(NULL); - } - sprintf(tmp, "%016llu.%016llu", llu(node.u), llu(node.l)); - return(tmp); -} - -triton_node_t triton_node_from_string(const char *s) -{ - int n; -#if SIZEOF_LONG_INT == 4 - uint64_t u; - uint64_t l; -#elif SIZEOF_LONG_INT == 8 - long long unsigned int u; - long long unsigned int l; -#else -# error Unexpected sizeof(long int) -#endif - - triton_node_t node = triton_node_null; - - n = sscanf(s, "%llu.%llu", &u, &l); - if (n != 2) - return triton_node_null; - - node.u = (uint64_t) u; - node.l = (uint64_t) l; - - return(node); -} - /* * Local Variables: * c-basic-offset: 4 diff --git a/code/src/common/node.h b/code/src/common/node.h index e9fdb7e..e1e492d 100644 --- a/code/src/common/node.h +++ b/code/src/common/node.h @@ -10,9 +10,8 @@ extern const triton_node_t triton_node_null; #define triton_node_equal(a,b) (triton_cmp_uint128(a,b)==0) -char* triton_node_to_string(triton_node_t node); - -triton_node_t triton_node_from_string(const char *s); +#define triton_node_to_string(a) triton_uint128_to_string(a) +#define triton_node_from_string(a) (triton_node_t)triton_uint128_from_string(a) int triton_cmp_node(triton_node_t a, triton_node_t b); diff --git a/code/src/replicated-osd/rosd-create.ae b/code/src/replicated-osd/rosd-create.ae index 918018d..e8aab64 100644 --- a/code/src/replicated-osd/rosd-create.ae +++ b/code/src/replicated-osd/rosd-create.ae @@ -42,7 +42,7 @@ static __blocking triton_ret_t triton_rpc_rosd_create(hg_handle_t handle) triton_rpc_rosd_create_out_t out; triton_rpc_rosd_create_in_t in; int ret = 0; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; int my_position; na_addr_t *addr_array = NULL; int got_addrs = 0; @@ -53,10 +53,11 @@ static __blocking triton_ret_t triton_rpc_rosd_create(hg_handle_t handle) triton_mercury_get_input(handle, &in, &out); - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, in.oid); + oid_str = triton_uint128_to_string(in.oid); triton_debug(triton_dbg_rosd, "triton_rpc_rosd_create() params: oid: %s, flags: %d, replication_factor: %d, niid: %llu\n", oid_str, in.flags, in.replication_factor, llu(in.niid)); + if (oid_str) free(oid_str); out.tret = TRITON_SUCCESS; @@ -187,15 +188,16 @@ static __blocking triton_ret_t rosd_create_do_work( /* forward request while simultaneously performing local work */ pwait { - pprivate char oid_str[TRITON_UINT128_STRLEN]; + pprivate char* oid_str; pbranch { if(my_position < replication_factor -1) { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd, "ROSD forwarding create of oid %s to %d'th server.\n", oid_str, my_position+1); + if (oid_str) free(oid_str); remote_tret = __remote_triton_rpc_rosd_create(next_addr, oid, replication_factor, @@ -204,8 +206,9 @@ static __blocking triton_ret_t rosd_create_do_work( } pbranch { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd, "ROSD local create of oid %s on %d'th server.\n", oid_str, my_position); + if (oid_str) free(oid_str); /* perform local operation */ local_tret = rosd_create_local_storage(oid, replication_factor, niid); } diff --git a/code/src/replicated-osd/rosd-remove.ae b/code/src/replicated-osd/rosd-remove.ae index 163e752..45b10d6 100644 --- a/code/src/replicated-osd/rosd-remove.ae +++ b/code/src/replicated-osd/rosd-remove.ae @@ -90,7 +90,7 @@ static __blocking triton_ret_t triton_rpc_rosd_remove(hg_handle_t handle) int i; na_addr_t *addr_array; na_addr_t next_addr; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; triton_ret_t tret; uint32_t replication_factor; @@ -108,7 +108,7 @@ static __blocking triton_ret_t triton_rpc_rosd_remove(hg_handle_t handle) triton_mercury_get_input(handle, &in, &out); - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, in.oid); + oid_str = triton_uint128_to_string(in.oid); triton_debug(triton_dbg_rosd, "triton_rpc_rosd_remove() params: oid: %s, flags: %d, replication_factor: %d, niid: %llu, position:%d\n", oid_str, @@ -116,6 +116,7 @@ static __blocking triton_ret_t triton_rpc_rosd_remove(hg_handle_t handle) in.replication_factor, llu(in.niid), in.expected_position); + if (oid_str) free(oid_str); /* * A replication factor of zero indicates this is the initial client @@ -244,14 +245,15 @@ static __blocking triton_ret_t rosd_remove_do_work( pwait { - pprivate char oid_str[TRITON_UINT128_STRLEN]; + pprivate char *oid_str; pbranch { /* forward on to peers if necessary */ if(my_position < replication_factor-1) { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd, "ROSD forwarding remove of oid %s to %d'th server.\n", oid_str, my_position+1); + if (oid_str) free(oid_str); remote_tret = __remote_triton_rpc_rosd_remove( next_addr, @@ -263,8 +265,9 @@ static __blocking triton_ret_t rosd_remove_do_work( } pbranch { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd, "ROSD local remove of oid %s on %d'th server.\n", oid_str, my_position); + if (oid_str) free(oid_str); /* perform local operation */ local_tret = rosd_remove_local_storage(oid, niid); } @@ -322,7 +325,7 @@ __blocking triton_ret_t __remote_triton_rpc_rosd_remove( int ret; triton_ret_t tret; int position; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; in.oid = oid; in.flags = flags; @@ -330,8 +333,9 @@ __blocking triton_ret_t __remote_triton_rpc_rosd_remove( in.niid = 0; /* TODO: generate real NIIDs */ in.expected_position = expected_position; - triton_uint128_to_string(oid_str, sizeof(oid_str), in.oid); + oid_str = triton_uint128_to_string(in.oid); triton_debug(triton_dbg_rosd, "__remote_triton_rpc_rosd_remove: post: oid=%s flags=%d rf=%d niid=%lld pos=%d\n", oid_str, in.flags, in.replication_factor, lld(in.niid), in.expected_position); + if (oid_str) free(oid_str); ret = HG_Forward(addr, rpc_rosd_remove_id, diff --git a/code/src/replicated-osd/rosd-txn-nr-cache.ae b/code/src/replicated-osd/rosd-txn-nr-cache.ae index 1210dcb..88be691 100644 --- a/code/src/replicated-osd/rosd-txn-nr-cache.ae +++ b/code/src/replicated-osd/rosd-txn-nr-cache.ae @@ -83,7 +83,7 @@ __blocking triton_ret_t txn_nr_cache_get(struct txn_nr_cache_entry **entry, uint struct triton_list_link* scratch; triton_ret_t tret; triton_ret_t tret_tmp; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; triton_mutex_lock(&txn_nr_mutex); assert(oid_table); @@ -112,10 +112,11 @@ __blocking triton_ret_t txn_nr_cache_get(struct txn_nr_cache_entry **entry, uint tmp_entry->txn_number++; *next_txn_number = tmp_entry->txn_number; } - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd_txncache, "returning cached entry for oid %s: txn_number: %llu\n", oid_str, llu(*next_txn_number)); + if(oid_str) free(oid_str); } else { @@ -188,10 +189,11 @@ __blocking triton_ret_t txn_nr_cache_get(struct txn_nr_cache_entry **entry, uint if(!(flags & ROSD_FLAG_COND_WRITE)) *next_txn_number = tmp_entry->txn_number; - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd_txncache, "generating new cache entry for oid %s: txn_number: %llu\n", oid_str, llu(*next_txn_number)); + if (oid_str) free(oid_str); } @@ -202,10 +204,10 @@ __blocking triton_ret_t txn_nr_cache_get(struct txn_nr_cache_entry **entry, uint { dead_entry = triton_list_get_entry(iterator, struct txn_nr_cache_entry, lru_link); - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, - dead_entry->oid); + oid_str = triton_uint128_to_string(dead_entry->oid); triton_debug(triton_dbg_rosd_txncache, "removing stale oid %s from cache\n", oid_str); + if(oid_str) free(oid_str); triton_list_del(&dead_entry->lru_link); triton_list_del(&dead_entry->hash_link); txn_nr_cache_destructor(dead_entry); /* decrements hash count too */ @@ -224,15 +226,14 @@ __blocking triton_ret_t txn_nr_cache_get(struct txn_nr_cache_entry **entry, uint void txn_nr_cache_put(struct txn_nr_cache_entry* entry) { - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; triton_mutex_lock(&txn_nr_mutex); assert(oid_table); entry->ref_count--; assert(entry->ref_count >= 0); - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, - entry->oid); + oid_str = triton_uint128_to_string(entry->oid); /* if ref count hits zero, then add to lru list as a candidate to be * dropped from the cache later @@ -249,7 +250,7 @@ void txn_nr_cache_put(struct txn_nr_cache_entry* entry) triton_debug(triton_dbg_rosd_txncache, "putting oid %s from cache, still active\n", oid_str); } - + if (oid_str) free(oid_str); triton_mutex_unlock(&txn_nr_mutex); return; } diff --git a/code/src/replicated-osd/rosd-write.ae b/code/src/replicated-osd/rosd-write.ae index 1ab6183..aba4f61 100644 --- a/code/src/replicated-osd/rosd-write.ae +++ b/code/src/replicated-osd/rosd-write.ae @@ -498,14 +498,15 @@ static __blocking triton_ret_t rosd_write_do_work( /* TODO: implement conditionals */ pwait { - pprivate char oid_str[TRITON_UINT128_STRLEN]; + pprivate char* oid_str; pbranch { /* forward on to peers if necessary */ if(my_position < replication_factor-1) { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd, "ROSD forwarding write of oid %s to %d'th server.\n", oid_str, my_position+1); + if (oid_str) free(oid_str); remote_tret = __remote_triton_rpc_rosd_write( next_addr, oid, oid_fork, buffer, size, offset, flags, replication_factor, @@ -514,8 +515,9 @@ static __blocking triton_ret_t rosd_write_do_work( } pbranch { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); triton_debug(triton_dbg_rosd, "ROSD local write of oid %s on %d'th server, transaction number %llu.\n", oid_str, my_position, llu(txn_number)); + if (oid_str) free(oid_str); /* perform local operation */ local_tret = rosd_write_local_storage(oid, oid_fork, txn_number, buffer, size, offset, flags); diff --git a/code/src/replicated-osd/rosd.ae b/code/src/replicated-osd/rosd.ae index eed96d2..fa72023 100644 --- a/code/src/replicated-osd/rosd.ae +++ b/code/src/replicated-osd/rosd.ae @@ -80,7 +80,7 @@ triton_ret_t triton_oid_to_addrs(uint128_t oid, uint32_t replication_factor, int triton_node_t* target_nodes; int i; triton_node_t self; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; *my_position = -1; @@ -112,7 +112,7 @@ triton_ret_t triton_oid_to_addrs(uint128_t oid, uint32_t replication_factor, int return(tret); } - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); for(i=0; i<replication_factor; i++) { @@ -141,6 +141,7 @@ triton_ret_t triton_oid_to_addrs(uint128_t oid, uint32_t replication_factor, int free(svr_string); } + if (oid_str) free(oid_str); free(target_nodes); free(node_array); diff --git a/code/src/system-state/system-state.ae b/code/src/system-state/system-state.ae index 5d0fd03..9929fd3 100644 --- a/code/src/system-state/system-state.ae +++ b/code/src/system-state/system-state.ae @@ -165,9 +165,9 @@ char* system_state_serialize(void) { size += strlen(entry->value) + - 33 + /* this is the string length of a node */ + INET6_ADDRSTRLEN + /* this is the string length of a node */ strlen(key_array[i].key) + - 8; /* delimiters */ + 8; /* delimiters */ } } @@ -199,7 +199,7 @@ char* system_state_serialize(void) node_str = triton_node_to_string(entry->node); ret = sprintf(&tmp[position], "<%s><%s><%s>", node_str, key_array[i].key, entry->value); - free(node_str); + if (node_str) free(node_str); position += ret; } } diff --git a/code/src/transactional-osd/tosd-fd-cache.ae b/code/src/transactional-osd/tosd-fd-cache.ae index 25eafda..cfda491 100644 --- a/code/src/transactional-osd/tosd-fd-cache.ae +++ b/code/src/transactional-osd/tosd-fd-cache.ae @@ -1,4 +1,4 @@ -#define _XOPEN_SOURCE 600 +#define _GNU_SOURCE #include <unistd.h> #include <sys/types.h> @@ -38,7 +38,7 @@ static __blocking void fd_cache_purge_internal(int watermark) struct fd_cache_entry* dead_entry; int ret; struct stat statbuf; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; triton_list_t purge_list; triton_list_init(&purge_list); @@ -84,11 +84,11 @@ static __blocking void fd_cache_purge_internal(int watermark) ret = aefile_ftruncate(dead_entry->fd, dead_entry->log_offset); if(ret != 0) { - triton_uint128_to_string(oid_str, - TRITON_UINT128_STRLEN, dead_entry->oid); + oid_str = triton_uint128_to_string(dead_entry->oid); triton_error_msg( "Failed to truncate object %s, continuing.\n", oid_str); + if (oid_str) free(oid_str); /* will show up as an error at write time, which * will make more sense */ @@ -182,7 +182,7 @@ __blocking void fd_cache_remove(uint128_t oid) struct fd_cache_entry* tmp_entry; char log_name [PATH_MAX]; int ret; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; triton_mutex_lock(&fd_mutex); @@ -216,8 +216,9 @@ __blocking void fd_cache_remove(uint128_t oid) } /* actually unlink the log file */ - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); sprintf(log_name, "%s/%s.dat", fd_cache_cosd_log_path, oid_str); + if (oid_str) free(oid_str); aefile_unlink(log_name); @@ -231,7 +232,7 @@ __blocking triton_ret_t fd_cache_get(struct fd_cache_entry **entry, uint128_t oi struct fd_cache_entry* new_entry; char log_name [PATH_MAX]; int posix_open_flags = O_RDWR|O_CREAT; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; struct stat statbuf; int rc, rc_tmp; @@ -279,8 +280,9 @@ __blocking triton_ret_t fd_cache_get(struct fd_cache_entry **entry, uint128_t oi posix_open_flags |= O_DSYNC; /* actually open the file */ - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, oid); + oid_str = triton_uint128_to_string(oid); sprintf(log_name, "%s/%s.dat", fd_cache_cosd_log_path, oid_str); + if (oid_str) free(oid_str); rc = aefile_open(log_name, posix_open_flags, S_IRUSR|S_IWUSR); if(rc < 0) @@ -403,7 +405,7 @@ static void fd_cache_destructor(struct fd_cache_entry* entry) { struct stat statbuf; int ret; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; /* we have done something wrong if this is still in use */ assert(entry->ref_count == 0); @@ -418,11 +420,11 @@ static void fd_cache_destructor(struct fd_cache_entry* entry) ret = ftruncate(entry->fd, entry->log_offset); if(ret < 0) { - triton_uint128_to_string(oid_str, - TRITON_UINT128_STRLEN, entry->oid); + oid_str = triton_uint128_to_string(entry->oid); triton_error_msg( "Failed to truncate object %s, continuing\n", oid_str); + if (oid_str) free(oid_str); /* will show up as an error at write time, which * will make more sense */ diff --git a/code/src/transactional-osd/transactional-osd.ae b/code/src/transactional-osd/transactional-osd.ae index ded0bd2..0d7b900 100644 --- a/code/src/transactional-osd/transactional-osd.ae +++ b/code/src/transactional-osd/transactional-osd.ae @@ -2160,7 +2160,7 @@ __blocking triton_ret_t tosd_dump(void) DBT log_map_key, missing_ver_key, ver_key, nd_key; DBT log_map_value, missing_ver_value, ver_value, nd_value; int c_get_flag = DB_SET_RANGE; - char oid_str[TRITON_UINT128_STRLEN]; + char *oid_str; struct niid_key nkey; struct niid_value nvalue; int ret; @@ -2233,10 +2233,11 @@ __blocking triton_ret_t tosd_dump(void) tret = triton_ret_from_bdb(ret); while (tret == TRITON_SUCCESS) { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, vk); + oid_str = triton_uint128_to_string(vk); printf("OID: %s\n", oid_str); printf("=============================================\n"); printf(" version: %llu\n", llu(vv)); + if (oid_str) free(oid_str); /* search for any missing versions */ printf(" missing versions: "); @@ -2332,9 +2333,10 @@ __blocking triton_ret_t tosd_dump(void) tret = triton_ret_from_bdb(ret); while (tret == TRITON_SUCCESS) { - triton_uint128_to_string(oid_str, TRITON_UINT128_STRLEN, nkey.oid); + oid_str = triton_uint128_to_string(nkey.oid); printf("OID: %s, NIID: %llu, op_type: %d, timestamp: %lld\n", oid_str, llu(nkey.niid), nkey.op_type, lld(nvalue.timestamp)); + if (oid_str) free(oid_str); ret = aebdb_cursor_get(niid_dbcp, &nd_key, &nd_value, DB_NEXT); tret = triton_ret_from_bdb(ret); } diff --git a/code/tests/placement/test-placement.c b/code/tests/placement/test-placement.c index 8fbab76..59ea08f 100644 --- a/code/tests/placement/test-placement.c +++ b/code/tests/placement/test-placement.c @@ -9,7 +9,7 @@ int main(int argc, char *argv[]) triton_node_t closest[3]; uint128_t oid; int i; - char id_str[TRITON_UINT128_STRLEN]; + char *id_str; servers[0] = triton_uint128_from_uint64(1); servers[1] = triton_uint128_from_uint64(UINT64_MAX/4ULL); @@ -32,15 +32,18 @@ int main(int argc, char *argv[]) printf("all servers:\n"); for(i=0; i<4; i++) { - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, servers[i]); + id_str = triton_uint128_to_string(servers[i]); printf("\t%s\n", id_str); + if (id_str) free(id_str); } - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, oid); + id_str = triton_uint128_to_string(oid); printf("closest to OID: %s:\n", id_str); + if (id_str) free(id_str); for(i=0; i<3; i++) { - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, closest[i]); + id_str = triton_uint128_to_string(closest[i]); printf("\t%s\n", id_str); + if (id_str) free(id_str); } printf("\n"); @@ -60,15 +63,18 @@ int main(int argc, char *argv[]) printf("all servers:\n"); for(i=0; i<4; i++) { - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, servers[i]); + id_str = triton_uint128_to_string(servers[i]); printf("\t%s\n", id_str); + if (id_str) free(id_str); } - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, oid); + id_str = triton_uint128_to_string(oid); printf("closest to OID: %s:\n", id_str); + if (id_str) free(id_str); for(i=0; i<3; i++) { - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, closest[i]); + id_str = triton_uint128_to_string(closest[i]); printf("\t%s\n", id_str); + if (id_str) free(id_str); } printf("\n"); @@ -94,15 +100,18 @@ int main(int argc, char *argv[]) printf("all servers:\n"); for(i=0; i<4; i++) { - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, servers[i]); + id_str = triton_uint128_to_string(servers[i]); printf("\t%s\n", id_str); + if (id_str) free(id_str); } - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, oid); + id_str = triton_uint128_to_string(oid); printf("closest to OID: %s:\n", id_str); + if (id_str) free(id_str); for(i=0; i<3; i++) { - triton_uint128_to_string(id_str, TRITON_UINT128_STRLEN, closest[i]); + id_str = triton_uint128_to_string(closest[i]); printf("\t%s\n", id_str); + if (id_str) free(id_str); } printf("\n"); diff --git a/code/tests/system-state/test-system-state.ae b/code/tests/system-state/test-system-state.ae index b345b46..3cd8288 100644 --- a/code/tests/system-state/test-system-state.ae +++ b/code/tests/system-state/test-system-state.ae @@ -13,7 +13,7 @@ __blocking int aesop_main(int argc, char **argv) triton_ret_t tret; triton_node_t node1; char* ser; - const char* load_test = "<0000000000000000.0000000000000001><testkey><foo>,<0000000000000000.0000000000000002><testkey><foo>,<0000000000000000.0000000000000001><testkeyB><foo>,<0000000000000000.0000000000000002><testkeyB><foo>"; + const char* load_test = "<::1><testkey><foo>,<::2><testkey><foo>,<::1><testkeyB><foo>,<::2><testkeyB><foo>"; char* val; triton_node_t* node_array; int node_array_count; diff --git a/code/tests/test-util.sh b/code/tests/test-util.sh index 1eca563..f6988e3 100644 --- a/code/tests/test-util.sh +++ b/code/tests/test-util.sh @@ -39,7 +39,7 @@ function test_start_servers () echo "# auto-generated host file from test pid $pid, $nservers servers" > /tmp/triton-hosts-$pid.txt for i in `seq $startport $endport` do - echo -e "0.$i\ttcp://localhost:$i" >> /tmp/triton-hosts-$pid.txt + echo -e "::$i\ttcp://localhost:$i" >> /tmp/triton-hosts-$pid.txt done # start daemons diff --git a/code/tests/triton-rm.sh b/code/tests/triton-rm.sh index 6a1cdf3..bf7b4f2 100755 --- a/code/tests/triton-rm.sh +++ b/code/tests/triton-rm.sh @@ -14,7 +14,7 @@ test_start_servers 4 15 # delete non-existent file ##################### -src/admin-tools/triton-rm --verbose --server $svr1 2.1 +src/admin-tools/triton-rm --verbose --server $svr1 ::2:0:0:0:1 if [ $? -ne 0 ]; then src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null test_fini @@ -24,8 +24,8 @@ fi # delete existing file w/ default replication ##################### -src/admin-tools/triton-touch $svr1 3.1 3 -src/admin-tools/triton-rm --verbose --server $svr1 3.1 +src/admin-tools/triton-touch $svr1 ::3:0:0:0:1 3 +src/admin-tools/triton-rm --verbose --server $svr1 ::3:0:0:0:1 if [ $? -ne 0 ]; then src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null test_fini @@ -35,8 +35,8 @@ fi # delete existing file w/ default replication ##################### -src/admin-tools/triton-touch $svr1 4.1 4 -src/admin-tools/triton-rm --verbose --server $svr1 4.1 +src/admin-tools/triton-touch $svr1 ::4:0:0:0:1 4 +src/admin-tools/triton-rm --verbose --server $svr1 ::4:0:0:0:1 if [ $? -ne 0 ]; then src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null test_fini diff --git a/code/tests/triton-start-test-servers.sh b/code/tests/triton-start-test-servers.sh index 275472d..932bba8 100755 --- a/code/tests/triton-start-test-servers.sh +++ b/code/tests/triton-start-test-servers.sh @@ -14,7 +14,7 @@ endport=`expr 3344 + $nservers - 1` echo "# auto-generated host file from test pid $pid, $nservers servers" > /tmp/triton-hosts-$pid.txt for i in `seq $startport $endport` do - echo -e "0.$i\ttcp://localhost:$i" >> /tmp/triton-hosts-$pid.txt + echo -e "::$i\ttcp://localhost:$i" >> /tmp/triton-hosts-$pid.txt done # start daemons diff --git a/code/tests/triton-touch.sh b/code/tests/triton-touch.sh index d15e52c..780c75b 100755 --- a/code/tests/triton-touch.sh +++ b/code/tests/triton-touch.sh @@ -14,7 +14,7 @@ test_start_servers 4 15 # actual test case ##################### -src/admin-tools/triton-touch $svr1 1.1 3 +src/admin-tools/triton-touch $svr1 ::1:0:0:0:1 3 if [ $? -ne 0 ]; then src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null test_fini hooks/post-receive --

1 0

C-Utils Repository branch, master, updated. 8acceabb88d7ba12c4d34099cfd23b19ee73c180
by noreply＠mcs.anl.gov 22 Nov '13

22 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, master has been updated via 8acceabb88d7ba12c4d34099cfd23b19ee73c180 (commit) via 783495cf74811fa0563af750ac1a07bb45d181bd (commit) via fc0c0934feba5c8c1b6624b3940458482106d448 (commit) from e11f958e01bea27318a047161add9842f297b0f4 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 8acceabb88d7ba12c4d34099cfd23b19ee73c180 Merge: 783495cf74811fa0563af750ac1a07bb45d181bd e11f958e01bea27318a047161add9842f297b0f4 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Fri Nov 22 16:43:14 2013 -0600 Merge branch 'master' of git://git.mcs.anl.gov/c-utils commit 783495cf74811fa0563af750ac1a07bb45d181bd Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Thu Nov 21 17:10:02 2013 -0600 change triton_uint128_{to,from}_string to work with ipv6 notation commit fc0c0934feba5c8c1b6624b3940458482106d448 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Oct 15 17:54:39 2013 -0500 this macro isn't supported on older automake versions so this change supports both ways ----------------------------------------------------------------------- Summary of changes: configure.ac | 2 +- src/triton-uint128.h | 57 +++++++++++++++++++++++++++++++++++++++++++++---- tests/test-uint128.c | 21 +++++++++++------- 3 files changed, 66 insertions(+), 14 deletions(-) Diff of changes: diff --git a/configure.ac b/configure.ac index 4a6007b..ccd1b0c 100644 --- a/configure.ac +++ b/configure.ac @@ -14,7 +14,7 @@ AC_CONFIG_HEADER([c-utils-config.h]) AC_PROG_CC AM_PROG_CC_C_O -AM_PROG_AR +m4_ifdef([AM_PROG_AR], [AM_PROG_AR]) LT_INIT([disable-shared]) AC_PROG_AWK AC_PROG_LN_S diff --git a/src/triton-uint128.h b/src/triton-uint128.h index 10184a6..48818e4 100644 --- a/src/triton-uint128.h +++ b/src/triton-uint128.h @@ -10,6 +10,10 @@ #include <stdio.h> #include <assert.h> #include <stdint.h> +#include <stdlib.h> +#include <arpa/inet.h> +#include <string.h> +#include <endian.h> /* unsigned 128 bit integer type */ typedef struct { uint64_t l; uint64_t u; } uint128_t; @@ -17,6 +21,7 @@ typedef struct { uint64_t l; uint64_t u; } uint128_t; #define triton_uint128_setzero(t) (t).l = 0, (t).u = 0 #define triton_uint128_iszero(t) ((t).l == 0 && (t).u == 0) #define triton_uint128_setmax(t) (t).l = UINT64_MAX, (t).u = UINT64_MAX +#define triton_uint128_ismax(t) ((t).l == UINT64_MAX && (t).u == UINT64_MAX) #define triton_uint128_set(from, to) ((to).l = (from).l, (to).u = (from).u) @@ -43,12 +48,54 @@ typedef struct { uint64_t l; uint64_t u; } uint128_t; #endif #endif -/* convert to string in uint64_t:uint64_t format (for now) */ -static inline void triton_uint128_to_string(char* buf, int buflen, uint128_t big) +/* convert uint128 to string which is in ipv6 notation, + * NULL on failure, + * user must free buffer when done */ +static inline char* triton_uint128_to_string(uint128_t big) { - assert(buflen >= TRITON_UINT128_STRLEN); - snprintf(buf, buflen, "%llu:%llu", llu(big.u), llu(big.l)); - return; + char *buf = malloc(INET6_ADDRSTRLEN); + const char *ptr; + + if (buf) + { + struct in6_addr ipv6_addr; + uint64_t *upper = (uint64_t *) &ipv6_addr.s6_addr[0]; + uint64_t *lower = (uint64_t *) &ipv6_addr.s6_addr[8]; + *upper = htobe64(big.u); + *lower = htobe64(big.l); + ptr = inet_ntop(AF_INET6, &ipv6_addr, buf, INET6_ADDRSTRLEN); + if (!ptr) + { + free(buf); /* free buffer on failure */ + buf = NULL; + } + } + + return buf; +} + +/* convert a string to a uint128_t, string is in ivp6 notation. + returns max oid on failure */ +static inline uint128_t triton_uint128_from_string (const char* str) +{ + struct in6_addr ipv6_addr; + uint128_t big; + int r; + uint64_t *upper = (uint64_t *) &ipv6_addr.s6_addr[0]; + uint64_t *lower = (uint64_t *) &ipv6_addr.s6_addr[8]; + + r = inet_pton(AF_INET6, str, &ipv6_addr); + if (r == 1) + { + big.u = be64toh(*upper); + big.l = be64toh(*lower); + } + else + { + triton_uint128_setmax(big); + } + + return big; } /* assign a 64 bit value to a uint128_t */ diff --git a/tests/test-uint128.c b/tests/test-uint128.c index 5d730b6..464368f 100644 --- a/tests/test-uint128.c +++ b/tests/test-uint128.c @@ -14,30 +14,35 @@ int main(int argc, char *argv[]) { uint128_t big; - char big_str[TRITON_UINT128_STRLEN]; + char *big_str; int testcase, failcount; tap_begin(testcase, failcount); big = triton_uint128_from_uint64(0); - triton_uint128_to_string(big_str, TRITON_UINT128_STRLEN, big); - tap("triton_uint128_from_uint64(0)", strcmp(big_str, "0:0") == 0, + big_str = triton_uint128_to_string(big); + tap("triton_uint128_from_uint64(0)", strcmp(big_str, "::") == 0, testcase, failcount); + if (big_str) free(big_str); big = triton_uint128_inc(big); - triton_uint128_to_string(big_str, TRITON_UINT128_STRLEN, big); - tap("triton_uint128_inc(0)", strcmp(big_str, "0:1") == 0, + big_str = triton_uint128_to_string(big); + tap("triton_uint128_inc(0)", strcmp(big_str, "::1") == 0, testcase, failcount); + if (big_str) free(big_str); big = triton_uint128_from_uint64(UINT64_MAX); - triton_uint128_to_string(big_str, TRITON_UINT128_STRLEN, big); + big_str = triton_uint128_to_string(big); tap("triton_uint128_from_uint64(UINT64_MAX)", big.l == UINT64_MAX, testcase, failcount); + if (big_str) free(big_str); big = triton_uint128_inc(big); - triton_uint128_to_string(big_str, TRITON_UINT128_STRLEN, big); - tap("triton_uint128_inc(UINT64_MAX)", strcmp(big_str, "1:0") == 0, + big_str = triton_uint128_to_string(big); + printf("%s\n", big_str); + tap("triton_uint128_inc(UINT64_MAX)", strcmp(big_str, "0:0:0:1::") == 0, testcase, failcount); + if (big_str) free(big_str); tap_end(testcase, failcount); hooks/post-receive -- C-Utils Repository

1 0

branch, master, updated. 591fb2a9b70847edc5e959bd609261a918847e52
by noreply＠mcs.anl.gov 21 Nov '13

21 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 591fb2a9b70847edc5e959bd609261a918847e52 (commit) from bea7f3a0442435408c1c9f8e393c1f37f05217f8 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 591fb2a9b70847edc5e959bd609261a918847e52 Author: Sumit Narayan <sumit(a)mcs.anl.gov> Date: Thu Nov 21 09:06:16 2013 -0600 Removed/Added missing lines that were causing the build to fail. ----------------------------------------------------------------------- Summary of changes: code/src/replicated-osd/rosd-write.ae | 1 - code/src/zeroconf/zeroconf.c | 1 + 2 files changed, 1 insertions(+), 1 deletions(-) Diff of changes: diff --git a/code/src/replicated-osd/rosd-write.ae b/code/src/replicated-osd/rosd-write.ae index 84c69a1..1ab6183 100644 --- a/code/src/replicated-osd/rosd-write.ae +++ b/code/src/replicated-osd/rosd-write.ae @@ -576,7 +576,6 @@ static __blocking triton_ret_t rosd_write_local_storage( char* buffer_offsets[1]; int tosd_flags = TOSD_FLAG_AUTO_TXN; triton_node_t self; - triton_string_t self_str; if(flags & ROSD_FLAG_TRUNC_WRITE) tosd_flags |= TOSD_FLAG_TRUNC_WRITE; diff --git a/code/src/zeroconf/zeroconf.c b/code/src/zeroconf/zeroconf.c index ce93117..6dfb7b0 100644 --- a/code/src/zeroconf/zeroconf.c +++ b/code/src/zeroconf/zeroconf.c @@ -1,6 +1,7 @@ #include "src/zeroconf/zeroconf.h" #include <triton-hash.h> +#include <triton-thread.h> #include "src/common/triton-error.h" #include <triton-list.h> hooks/post-receive --

1 0

branch, master, updated. bea7f3a0442435408c1c9f8e393c1f37f05217f8
by noreply＠mcs.anl.gov 15 Nov '13

15 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via bea7f3a0442435408c1c9f8e393c1f37f05217f8 (commit) from 2083646d75bc4d9a91a48f41bbca663dabdfa191 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit bea7f3a0442435408c1c9f8e393c1f37f05217f8 Author: Sumit Narayan <sumit(a)mcs.anl.gov> Date: Fri Nov 15 12:48:13 2013 -0600 Updated the INSTALL file with information on how to build Mercury with SSM. ----------------------------------------------------------------------- Summary of changes: code/INSTALL | 49 +++++++++++++++++++++++++++++++++++++++++-------- 1 files changed, 41 insertions(+), 8 deletions(-) Diff of changes: diff --git a/code/INSTALL b/code/INSTALL index 54d62c6..a43d627 100644 --- a/code/INSTALL +++ b/code/INSTALL @@ -16,21 +16,54 @@ network transport. make make install +* SSM over TCP (Get latest SSM libraries from: + https://wiki.mcs.anl.gov/asg-private/index.php/Main_Page#SSM) + + - Build base SSM library: + cd libssm_ref-0.6.6-r2263 + ./configure --prefix=<SSM_INSTALL_PREFIX> + make + make install + + - Build TCP Library + cd libssmptcp_ref-0.6.6-r2264 + CFLAGS=-I<SSM_INSTALL_PREFIX>/include ./configure --prefix=<SSM_INSTALL_PREFIX> + make + make install + * Mercury - (note that Mercury _must_ be built with both BMI and Boost support) + NOTE: - Mercury _must_ be built with Boost support. + - It _must_ be built with either BMI or SSM, or both. + git clone git://git.mcs.anl.gov/radix/mercury cd mercury git submodule init git submodule update mkdir build cd build - cmake -DNA_USE_BMI:BOOL=ON \ - -DBMI_INCLUDE_DIR:PATH=<BMI_INSTALL_PREFIX>/include \ - -DBMI_LIBRARY:FILEPATH=<BMI_INSTALL_PREFIX>/lib/libbmi.a \ - -DMERCURY_USE_BOOST_PP:BOOL=ON \ - -DMERCURY_ENABLE_VERBOSE_ERROR:BOOL=OFF \ - -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo \ - -DCMAKE_INSTALL_PREFIX=/tmp/mercury-install ../ + + - When using BMI: + + cmake -DNA_USE_BMI:BOOL=ON \ + -DNA_USE_SSM:BOOL=ON \ + -DBMI_INCLUDE_DIR:PATH=<BMI_INSTALL_PREFIX>/include \ + -DBMI_LIBRARY:FILEPATH=<BMI_INSTALL_PREFIX>/lib/libbmi.a \ + -DMERCURY_USE_BOOST_PP:BOOL=ON \ + -DMERCURY_ENABLE_VERBOSE_ERROR:BOOL=OFF \ + -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo \ + -DCMAKE_INSTALL_PREFIX=/tmp/mercury-install ../ + + - When using SSM (over TCP): + + cmake -DNA_USE_SSM:BOOL=ON \ + -DSSM_INCLUDE_DIR:PATH=<SSM_INSTALL_PREFIX>/include \ + -DSSM_LIBRARY:FILEPATH=<SSM_INSTALL_PREFIX>/lib/libssm.a \ + -SSM_LIBRARY_SSMPTCP:FILEPATH=<SSM_INSTALL_PREFIX>/lib/libssmptcp.a \ + -DMERCURY_USE_BOOST_PP:BOOL=ON \ + -DMERCURY_ENABLE_VERBOSE_ERROR:BOOL=OFF \ + -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo \ + -DCMAKE_INSTALL_PREFIX=/tmp/mercury-install ../ + make make install hooks/post-receive --

1 0

branch, master, updated. 2083646d75bc4d9a91a48f41bbca663dabdfa191
by noreply＠mcs.anl.gov 11 Nov '13

11 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 2083646d75bc4d9a91a48f41bbca663dabdfa191 (commit) from 19d1dcf6336f8414d51155d24ae71152ebf5467a (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 2083646d75bc4d9a91a48f41bbca663dabdfa191 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 11 16:56:30 2013 -0500 convert other test scripts to use $srcdir ----------------------------------------------------------------------- Summary of changes: code/configure.ac | 6 -- code/tests/triton-cp.sh.in | 55 -------------------- code/tests/{triton-ls.sh.in => triton-ls.sh} | 8 ++- ...op-multi-svr.sh.in => triton-noop-multi-svr.sh} | 8 ++- code/tests/{triton-noop.sh.in => triton-noop.sh} | 8 ++- code/tests/{triton-rm.sh.in => triton-rm.sh} | 8 ++- ...tem-state.sh.in => triton-show-system-state.sh} | 8 ++- code/tests/{triton-touch.sh.in => triton-touch.sh} | 8 ++- 8 files changed, 30 insertions(+), 79 deletions(-) delete mode 100755 code/tests/triton-cp.sh.in rename code/tests/{triton-ls.sh.in => triton-ls.sh} (80%) rename code/tests/{triton-noop-multi-svr.sh.in => triton-noop-multi-svr.sh} (80%) rename code/tests/{triton-noop.sh.in => triton-noop.sh} (80%) rename code/tests/{triton-rm.sh.in => triton-rm.sh} (91%) rename code/tests/{triton-show-system-state.sh.in => triton-show-system-state.sh} (95%) rename code/tests/{triton-touch.sh.in => triton-touch.sh} (80%) Diff of changes: diff --git a/code/configure.ac b/code/configure.ac index b1ed0ed..025950f 100755 --- a/code/configure.ac +++ b/code/configure.ac @@ -128,12 +128,6 @@ AC_ARG_ENABLE(strict, ]) AC_CONFIG_FILES([Makefile]) -AC_CONFIG_FILES([tests/triton-noop.sh],[chmod +x tests/triton-noop.sh]) -AC_CONFIG_FILES([tests/triton-touch.sh],[chmod +x tests/triton-touch.sh]) -AC_CONFIG_FILES([tests/triton-ls.sh],[chmod +x tests/triton-ls.sh]) -AC_CONFIG_FILES([tests/triton-rm.sh],[chmod +x tests/triton-rm.sh]) -AC_CONFIG_FILES([tests/triton-noop-multi-svr.sh],[chmod +x tests/triton-noop-multi-svr.sh]) -AC_CONFIG_FILES([tests/triton-show-system-state.sh],[chmod +x tests/triton-show-system-state.sh]) dnl Make doc output dir AC_CONFIG_COMMANDS([mkdir], [$MKDIR_P doc doc/fig doc/resilience doc/security]) diff --git a/code/tests/triton-cp.sh.in b/code/tests/triton-cp.sh.in deleted file mode 100755 index a4d6ece..0000000 --- a/code/tests/triton-cp.sh.in +++ /dev/null @@ -1,55 +0,0 @@ -#!/bin/bash - -if [ -z $srcdir ]; then - echo srcdir variable not set. - exit 1 -fi -source $srcdir/test/test-util.sh - -# initialize test with 60 second timeout -test_init 60 - -# start 4 servers with 15 second wait -test_start_servers 4 15 - -# actual test case -##################### -dd if=/dev/urandom of=/tmp/8-$$.dat bs=1K count=8 - -src/admin-tools/triton-cp $svr1 posix:/tmp/8-$$.dat 1.8 3 -if [ $? -ne 0 ]; then - src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null - test_fini - exit 1 -fi - -src/admin-tools/triton-cp $svr1 1.8 posix:/tmp/8-$$.dat.check -if [ $? -ne 0 ]; then - src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null - test_fini - exit 1 -fi - -src/admin-tools/triton-rm --server $svr1 1.8 -if [ $? -ne 0 ]; then - src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null - test_fini - exit 1 -fi - -diff /tmp/8-$$.dat /tmp/8-$$.dat.check -if [ $? -ne 0 ]; then - echo ERROR: data copied in and out of Triton does not match. - src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null - test_fini - exit 1 -fi - -##################### - -# tear down -src/admin-tools/triton-shutdown-all-servers $svr1 &> /dev/null - -test_fini - -exit 0 diff --git a/code/tests/triton-ls.sh.in b/code/tests/triton-ls.sh similarity index 80% rename from code/tests/triton-ls.sh.in rename to code/tests/triton-ls.sh index 5eeec5a..2d3f687 100755 --- a/code/tests/triton-ls.sh.in +++ b/code/tests/triton-ls.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # intialize test with 60 second timeout test_init 60 diff --git a/code/tests/triton-noop-multi-svr.sh.in b/code/tests/triton-noop-multi-svr.sh similarity index 80% rename from code/tests/triton-noop-multi-svr.sh.in rename to code/tests/triton-noop-multi-svr.sh index 6778d55..b6956c9 100755 --- a/code/tests/triton-noop-multi-svr.sh.in +++ b/code/tests/triton-noop-multi-svr.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # initialize test with 60 second timeout test_init 60 diff --git a/code/tests/triton-noop.sh.in b/code/tests/triton-noop.sh similarity index 80% rename from code/tests/triton-noop.sh.in rename to code/tests/triton-noop.sh index 68d963b..3dac233 100755 --- a/code/tests/triton-noop.sh.in +++ b/code/tests/triton-noop.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # initalize test with 60 second timeout test_init 60 diff --git a/code/tests/triton-rm.sh.in b/code/tests/triton-rm.sh similarity index 91% rename from code/tests/triton-rm.sh.in rename to code/tests/triton-rm.sh index 09da798..6a1cdf3 100755 --- a/code/tests/triton-rm.sh.in +++ b/code/tests/triton-rm.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # initialize test with 60 second timeout test_init 60 diff --git a/code/tests/triton-show-system-state.sh.in b/code/tests/triton-show-system-state.sh similarity index 95% rename from code/tests/triton-show-system-state.sh.in rename to code/tests/triton-show-system-state.sh index 02823f6..2e9c0f1 100755 --- a/code/tests/triton-show-system-state.sh.in +++ b/code/tests/triton-show-system-state.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # We default to 4 servers if no input is given. if [[ -n "$1" ]] && (( $1 >= 0 )); then diff --git a/code/tests/triton-touch.sh.in b/code/tests/triton-touch.sh similarity index 80% rename from code/tests/triton-touch.sh.in rename to code/tests/triton-touch.sh index 8f5fd60..d15e52c 100755 --- a/code/tests/triton-touch.sh.in +++ b/code/tests/triton-touch.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # initialize test with 60 second timeout test_init 60 hooks/post-receive --

1 0

branch, master, updated. 19d1dcf6336f8414d51155d24ae71152ebf5467a
by noreply＠mcs.anl.gov 11 Nov '13

11 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 19d1dcf6336f8414d51155d24ae71152ebf5467a (commit) via db106253d112a09c95c1ebcc3f96b213250e4cbb (commit) via 16a2e7540807ae11b261f4846b3e35a6af0451cf (commit) from 14d83a58e77c028f4eadc0219c8c329cd3f958ef (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 19d1dcf6336f8414d51155d24ae71152ebf5467a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 11 16:32:16 2013 -0500 update triton-cp.sh test to use $srcdir commit db106253d112a09c95c1ebcc3f96b213250e4cbb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 11 16:31:40 2013 -0500 update server daemon command line docs commit 16a2e7540807ae11b261f4846b3e35a6af0451cf Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 11 16:30:43 2013 -0500 note on how to run individual tests ----------------------------------------------------------------------- Summary of changes: code/INSTALL | 8 +++++++- code/configure.ac | 1 - code/tests/{triton-cp.sh.in => triton-cp.sh} | 8 +++++--- code/tests/triton-cp.sh.in | 8 +++++--- 4 files changed, 17 insertions(+), 8 deletions(-) copy code/tests/{triton-cp.sh.in => triton-cp.sh} (91%) Diff of changes: diff --git a/code/INSTALL b/code/INSTALL index 3029dab..54d62c6 100644 --- a/code/INSTALL +++ b/code/INSTALL @@ -57,6 +57,10 @@ Building Triton * (optional) run tests make check +* (optional) run individual test scripts + # this example runs only the triton-cp test script + make TESTS=tests/triton-cp.sh check + Running an example system ========================= @@ -68,5 +72,7 @@ something like this: Then, to run triton server, issue the following command: -$ src/server/triton-server hostfile.txt tcp://localhost:3344 +$ src/server/triton-server --hostfile hostfile.txt --listenaddr tcp://localhost:3344 $ src/admin-tools/triton-noop tcp://localhost:3344 + +You can use the --daemonize option to have the server run in the background. diff --git a/code/configure.ac b/code/configure.ac index a1c1f09..b1ed0ed 100755 --- a/code/configure.ac +++ b/code/configure.ac @@ -130,7 +130,6 @@ AC_ARG_ENABLE(strict, AC_CONFIG_FILES([Makefile]) AC_CONFIG_FILES([tests/triton-noop.sh],[chmod +x tests/triton-noop.sh]) AC_CONFIG_FILES([tests/triton-touch.sh],[chmod +x tests/triton-touch.sh]) -AC_CONFIG_FILES([tests/triton-cp.sh],[chmod +x tests/triton-cp.sh]) AC_CONFIG_FILES([tests/triton-ls.sh],[chmod +x tests/triton-ls.sh]) AC_CONFIG_FILES([tests/triton-rm.sh],[chmod +x tests/triton-rm.sh]) AC_CONFIG_FILES([tests/triton-noop-multi-svr.sh],[chmod +x tests/triton-noop-multi-svr.sh]) diff --git a/code/tests/triton-cp.sh.in b/code/tests/triton-cp.sh similarity index 91% copy from code/tests/triton-cp.sh.in copy to code/tests/triton-cp.sh index 8e184d0..cb93fb6 100755 --- a/code/tests/triton-cp.sh.in +++ b/code/tests/triton-cp.sh @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/tests/test-util.sh # initialize test with 60 second timeout test_init 60 diff --git a/code/tests/triton-cp.sh.in b/code/tests/triton-cp.sh.in index 8e184d0..a4d6ece 100755 --- a/code/tests/triton-cp.sh.in +++ b/code/tests/triton-cp.sh.in @@ -1,8 +1,10 @@ #!/bin/bash -abs_srcdir=@abs_srcdir@ - -source $abs_srcdir/test-util.sh +if [ -z $srcdir ]; then + echo srcdir variable not set. + exit 1 +fi +source $srcdir/test/test-util.sh # initialize test with 60 second timeout test_init 60 hooks/post-receive --

1 0

branch, master, updated. 154b13017653474b4fa28812a84a135769d3b25e
by noreply＠mcs.anl.gov 10 Nov '13

10 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 154b13017653474b4fa28812a84a135769d3b25e (commit) from c02e99faccdaf94e1ec90775e3f1a880aa7b380f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 154b13017653474b4fa28812a84a135769d3b25e Author: Sumit Narayan <sumit(a)mcs.anl.gov> Date: Sun Nov 10 18:24:18 2013 -0600 NA SSM Report ----------------------------------------------------------------------- Summary of changes: reports/na_ssm_update.txt | 172 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 172 insertions(+), 0 deletions(-) create mode 100644 reports/na_ssm_update.txt Diff of changes: diff --git a/reports/na_ssm_update.txt b/reports/na_ssm_update.txt new file mode 100644 index 0000000..f0401c7 --- /dev/null +++ b/reports/na_ssm_update.txt @@ -0,0 +1,172 @@ +Network Abstraction in Mercury over Single-Sided Messaging (SSM) Library +======================================================================== + +Mercury provides an asynchronous RPC interface that is specifically +designed for high-performance computing systems. It not only allows +asynchronous transfer of parameters and execution requests, but also +provides direct support for large data arguments. Mercury is build on +top of a Network Abstraction layer that abstracts the network protocol +that is being used for data transfer from the user. This allows a +variety of transport protocols to be integrated with Mercury as a +Network Abstraction plugin, and without requiring any change required +in the user’s source code. + +In the past (ENTER COUNT) months, we have successfully been able to +build a single-sided messaging (SSM) plugin for the Network +Abstraction layer that communicates with other clients using SSM's TCP +libraries. Since SSM is an abstraction layer on its own, it can also +support multiple transport protocol. At the time of this document, we +only had access to SSM's TCP libraries and hence our network +communication was done over TCP protocol. Given the abstraction that +SSM provides, it will require minimal effort to run this code over +faster networks such as Infiniband (SSM's Infiniband libraries are +available in beta, but we did not make any attempt to use it yet.) + +We have successfully been able to handle most of the data transfer +required for communication from Triton's perspective. Unexpected and +expected messages could be received on both, servers and clients. We +have also successfully accessed remote memory using the ssm_get() and +ssm_put() interfaces exported by the SSM library. + +Below is the list of operations that we can successfully do over SSM +today: + +- Unexpected send +- Unexpected receive +- Message send (expected) +- Message receive (expected), with remote offset aligned with the + tagged message offset + + +Network Abstraction (NA) Layer +============================== + +Network Abstraction layer exports a set of APIs to be used by Mercury +for data transfer. These APIs were influenced by Portals, although it +will be hard to match them one-on-one. These interfaces are async, in +that they require the caller to provide a callback function which will +be called upon completion of the requested operation. + +NA-SSM Design +============= + +Unexpected Buffers +------------------ + +The NA-SSM layer maintains a set of buffers that is used to receive +and store any unexpected messages. The size of this buffer can be +varied by modifying the NA_SSM_UNEXPECTED_BUFFERCOUNT macro. Its +value will depend on the rate at which we expect the unexpected +requests to arrive on the server, and the rate at which we can copy +the buffer to the server's memory region. + +Known Issues +============ + +We have not been able to do a pipeline transfer of data when using +remote offsets. Our use-case is described below: A server exposes a +large segment of its memory to be copied to the client by posting that +memory segment with a tag and SSM_POST_STATIC flag. The client, to +copy this large segment of memory can allocate multiple buffers of +smaller sizes and do ssm_get() over the remote memory providing remote +offsets. + +In our experiments, we failed to obtain the memory when providing +remote offsets. We received a no matching buffer error for the tag +that we used. + +Design complexity and our solution +================================== + +We have a known design complexity when interfacing Mercury's Network +Abstraction layer with the SSM library. + +SSM posts a memory region to a match entry based on match bits +provided by the user. These match bits, which are 64-bit values, are +set by the user that tags a memory region for a recipient. SSM then +matches any incoming messages using a simple semantics of matching +these match bits. The memory regions are queued in a list of match +entries. If there are more than one memory region posted with the +same match bits, the regions are put in a queue called match list. +New posts are always appended to the end of this queue. When a +message is received by SSM, it is compared against the posted match +entries. If the tags match, the message is accepted, and the transfer +is done; otherwise, either the next match entry if exists is compared, +or else, the request fails. + +The issue we see in our Network Abstraction layer when multiple memory +regions that are tagged with the same match bits is when memory +regions of different sizes are registed with post for the same match +bits. These memory regions are queued in match list in the order that +they were posted, or in other words, originally received by the +Network Abstraction layer. Note that the new memory region is always +posted to the end of the list. The get or put request however, can +arrive in any order. When an order is received, SSM only looks at the +first entry (top) of the list. If the size of the request is too +large to fit in the memory region, or if the memory region is too +large to fit in the sender’s message, the request fails. Since we +have no control on the order in which the put or get request can +arrive, there is a likelihood that a request could fail under some +scenarios because of the size error. These scenarios are explained in +more detail below. + +When multiple memory regions are posted for transfer, the +corresponding put or get requests can arrive in any order. Let us +consider the simple case of only two regions MR1 and MR2 posted, with +size of MR1 less than the size of MR2. + +... sizeof(MR1) < sizeof(MR2) + +The two transfer requests that arrive on the server can then fall in +one of the two categories below: + +a) First request smaller than the second request: +... sizeof(TransferRequest1) < sizeof(TransferRequest2) + +If the size of the first request received for the tag is smaller than +the size of the second request, it implies the order in which the +requests were received is the same as the order in which the memory +regions were posted. In this case, no operations will fail. + +b) First request greater than the second request: +... sizeof(TransferRequest) > sizeof(TransferRequest2) + +If the size of the first request is greater than the second request, +the first transfer operation will fail. This is because the buffer +posted is smaller than the buffer received. The second transfer order +will however pass. + +... sizeof(MR1) > sizeof(MR1) + +a) Smaller requests arrive before the bigger request +... sizeof(TransferRequest1) < sizeof(TransferRequest2) + +In this case now, the first request will succeed, but the second +request will fail. + +b) Larger request arrives before the smaller request +... sizeof(TransferRequest) > sizeof(TransferRequest2) + +Since this order is the same as the order in which the memory was +posted, the requests will succeed for both transfer request. + +As we can see above, the outcome depends entirely on the +synchronization of the order in which the transfer request is +received. + +To make this issue deterministic, we have come up with the following +solution to be implemented at SSM plugin of the Network Abstraction +layer. + +We plan to create a common tag that will have memory regions of +maximum allowable transfer size posted. At the NA-SSM layer, when +registering a memory region for transfer, if we observe that there +already exists a region already posted for the same tag, instead of +queuing the new memory region behind the previous request, we create a +new request with maximum buffer size on the common tag. In a local +match table, we also keep track of the original memory pointers and +the mapped memory points. On the client size, if a request for memory +transfer fails, we reissue the request with the "common tag". Since a +large request was already posted on the server, the second attempt for +transfer will succeed. + hooks/post-receive --

1 0

branch, master, updated. 14d83a58e77c028f4eadc0219c8c329cd3f958ef
by noreply＠mcs.anl.gov 07 Nov '13

07 Nov '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 14d83a58e77c028f4eadc0219c8c329cd3f958ef (commit) from cebc57c6e7cff2e2c3fe4a14fef44c2b3cddad06 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 14d83a58e77c028f4eadc0219c8c329cd3f958ef Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Thu Nov 7 20:50:56 2013 -0600 And again I forget a file At least I'm checking *after* committing. Next time I'll check *before* :-) ----------------------------------------------------------------------- Summary of changes: code/doc/security/Makefile.subdir | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) create mode 100644 code/doc/security/Makefile.subdir Diff of changes: diff --git a/code/doc/security/Makefile.subdir b/code/doc/security/Makefile.subdir new file mode 100644 index 0000000..76135fc --- /dev/null +++ b/code/doc/security/Makefile.subdir @@ -0,0 +1,2 @@ +ASCIIDOCSRC += doc/security/proposal.txt + hooks/post-receive --

1 0

Re: [triton-commits] branch, master, updated. 04d394433752fa2a4d19894b4abe13b570be7832
by Dries Kimpe 07 Nov '13

07 Nov '13

Give it a shot now... There seem to be some asciidoc issues (version incompatibilities?) but all the build rules and steps are in place. I tried in-tree and VPATH build, and made it so that the PDFs get included in make dist (didn't do distcheck) so users won't need to have dblatex/asciidoc installed to build triton. Dries * Phil Carns <carns(a)mcs.anl.gov> [2013-11-07 10:55:26]: > Thanks Dries. I think there might be some files missing from the commit, > though? Here is what I get from the prepare script after pulling these > updates: > $ ./prepare.sh > Regenerating build files... > automake: error: cannot open < doc/Makefile.subdir: No such file or > directory > autoreconf: automake failed with exit status: 1 > -Phil > On 11/06/2013 08:28 PM, mysql vizuser wrote: > >This is an automated email from the git hooks/post-receive script. It was > >generated because a ref change was pushed to the repository containing > >the project "". > >The branch, master has been updated > > via 04d394433752fa2a4d19894b4abe13b570be7832 (commit) > > via cdedeec0f8e2874bfe19f47483e808ceedcb4137 (commit) > > from e5e9a52596c87f9db175a8c3a03cdb3b11eba996 (commit) > >Those revisions listed above that are new to this repository have > >not appeared on any other notification email; so we list those > >revisions in full, below. > >- Log ----------------------------------------------------------------- > >commit 04d394433752fa2a4d19894b4abe13b570be7832 > >Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> > >Date: Wed Nov 6 19:28:09 2013 -0600 > > WOrk on restoring doc rules > > Think this is OK, but can't test since my local a2x/dblatex seems > > broken... > > If somebody could test, that would be great. > >commit cdedeec0f8e2874bfe19f47483e808ceedcb4137 > >Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> > >Date: Wed Nov 6 18:20:34 2013 -0600 > > Update handle-ditaa.py for python3 > > Tested with 2.7.5 and 3.2.5 > >----------------------------------------------------------------------- > >Summary of changes: > > code/Make.rules | 29 +++++++++++++++++++++++++++++ > > code/Makefile.am | 7 +++++++ > > code/configure.ac | 8 ++++++++ > > code/maint/handle-ditaa.py | 10 +++++----- > > 4 files changed, 49 insertions(+), 5 deletions(-) > >Diff of changes: > >diff --git a/code/Make.rules b/code/Make.rules > >index cc48332..e4f4924 100644 > >--- a/code/Make.rules > >+++ b/code/Make.rules > >@@ -33,3 +33,32 @@ dirname = $(patsubst %/,%,$(dir $(1))) > > mkdir -p $(call dirname,$@) > > CC="$(CC)" CFLAGS="$(CFLAGS)" $(srcdir)/maint/depend-ae.sh $(call dirname,$@) $< > $@ > >+# > >+# Asciidoc rules > >+# > >+PDFDOCS = > >+ > >+PDFDOCS += $(patsubst %.txt, %.pdf, $(ASCIIDOCSRC)) > >+ > >+# Ensure docs we built get cleaned by automake > >+# Docs are included in the distribution, so only remove when doing maintainerclean > >+MAINTAINERCLEANFILES += $(PDFDOCS) > >+ > >+# Include docs in distribution so user doesn't have to have DITAA or ASCIIDOC > >+EXTRA_DIST += $(PDFDOCS) > >+ > >+.PHONY: doc > >+doc: $(PDFDOCS) > >+ > >+%.pdf: %.ditta_txt $(builddir)/doc/fig > >+ $(A2X) -L -v -v -a buildpath=$(abs_builddir)/doc --dblatex-opts "-P latex.output.revhistory=0" -a data-uri -a icons -a toc -a max-width=55em $< > >+ echo #test -n "$(A2X)" || echo "Need A2X to regenerate documentation" && false > >+ > >+#$(builddir)/doc/fig: $(abs_srcdir)/doc/fig > >+# ln -s $< $@ > >+ > >+.INTERMEDIATE: %.ditta_txt > >+%.ditta_txt: %.txt > >+ $(srcdir)/maint/handle-ditaa.py -j $(srcdir)/maint/ditaa/ditaa.jar -d $(dir $@) $< >$@ > >+ > >+ > >diff --git a/code/Makefile.am b/code/Makefile.am > >index 6c37db0..7309ae8 100644 > >--- a/code/Makefile.am > >+++ b/code/Makefile.am > >@@ -11,6 +11,7 @@ TESTS = > > check_PROGRAMS = > > EXTRA_PROGRAMS = > > CLEANFILES = $(bin_SCRIPTS) > >+MAINTAINERCLEANFILES = > > EXTRA_DIST = > > BUILT_SOURCES = > > AM_CTAGSFLAGS=--excmd=pattern -B --extra=+f --langmap=c:+.ae.hae -I pwait,pbranch > >@@ -34,6 +35,10 @@ src_libasg_a_SOURCES = > > AE_SRC = > > AE_HDR = > >+A2X=@A2X@ > >+ > >+ASCIIDOCSRC = > >+ > > include Make.rules > > include $(top_srcdir)/src/common/Makefile.subdir > >@@ -49,5 +54,7 @@ include $(top_srcdir)/src/replicated-osd/Makefile.subdir > > include $(top_srcdir)/src/placement/Makefile.subdir > > include $(top_srcdir)/tests/Makefile.subdir > >+include $(top_srcdir)/doc/Makefile.subdir > >+ > > -include $(AE_SRC:.ae=.ae.d) > > -include $(AE_HDR:.hae=.hae.d) > >diff --git a/code/configure.ac b/code/configure.ac > >index cb9c639..def6527 100755 > >--- a/code/configure.ac > >+++ b/code/configure.ac > >@@ -22,6 +22,11 @@ AC_PROG_CXX > > AC_PROG_CXXCPP > > AC_PROG_RANLIB > >+AC_PROG_MKDIR_P > >+ > >+AC_CHECK_PROGS([A2X], [a2x], []) > >+AC_CHECK_PROGS([DBLATEX],[dblatex],[]) > >+ > > AC_REQUIRE_CPP > > AC_CHECK_SIZEOF([long int]) > >@@ -129,4 +134,7 @@ AC_CONFIG_FILES([tests/triton-rm.sh],[chmod +x tests/triton-rm.sh]) > > AC_CONFIG_FILES([tests/triton-noop-multi-svr.sh],[chmod +x tests/triton-noop-multi-svr.sh]) > > AC_CONFIG_FILES([tests/triton-show-system-state.sh],[chmod +x tests/triton-show-system-state.sh]) > >+dnl Make doc output dir > >+AC_CONFIG_COMMANDS([mkdir], [$MKDIR_P doc doc/fig]) > >+ > > AC_OUTPUT > >diff --git a/code/maint/handle-ditaa.py b/code/maint/handle-ditaa.py > >index e822efc..aad9a62 100755 > >--- a/code/maint/handle-ditaa.py > >+++ b/code/maint/handle-ditaa.py > >@@ -29,8 +29,8 @@ def main(document, ditaaPath, imageDir): > > worklist[filename] = "" > > elif line.strip() == "[DITAA END]": > > fd, asciiname = tempfile.mkstemp( > >- suffix = ".txt", prefix="asciiart-", dir="/tmp") > >- a = os.fdopen(fd, "w+b") > >+ suffix = ".txt", prefix="asciiart-", dir="/tmp") > >+ a = os.fdopen(fd, "w+b") > > a.write(worklist[filename]) > > a.close() > > parsing = False > >@@ -42,7 +42,7 @@ def main(document, ditaaPath, imageDir): > > else: > > parsed.append(line) > >- print "".join(parsed) > >+ print ("".join(parsed)) > > if __name__ == "__main__": > >@@ -57,9 +57,9 @@ if __name__ == "__main__": > > ditaaPath = v > > if imageDir == None: > >- print "usage: [-d <image directory>] [-j <path to ditaa jar>]\n" > >+ print ("usage: [-d <image directory>] [-j <path to ditaa jar>]\n") > > sys.exit(1) > >- > >+ > > if len(args) == 1: > > document = open(args[0], "r").readlines() > > else: > >hooks/post-receive > >-- > >_______________________________________________ > >triton-commits mailing list > >triton-commits(a)lists.mcs.anl.gov > >https://lists.mcs.anl.gov/mailman/listinfo/triton-commits > _______________________________________________ > triton-commits mailing list > triton-commits(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/triton-commits

1 0