March 2015 - Triton-commits - mailman3.cels.anl.gov

branch, master, updated. fd43fbe54cacc9394d30a35c289d37e8753b2ff2
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via fd43fbe54cacc9394d30a35c289d37e8753b2ff2 (commit) from 7e5bfcb3cee9e69803e0615ecb9f29528ef6f5f2 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit fd43fbe54cacc9394d30a35c289d37e8753b2ff2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 14:06:58 2015 -0400 rough draft material for 2.1.6a (localstore) ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/2.1.6a-localstore-perf.pdf} | Bin 8200 -> 8200 bytes .../2015.03.30/2.1.6a-localstore-perf.pdf.svg.emf | Bin 0 -> 125832 bytes .../2015.03.30/2.1.6a-localstore.txt | 40 ++++++++++++++++++++ 3 files changed, 40 insertions(+), 0 deletions(-) copy reports/{triton-localstore-convergence/figs/hoss-ssd/hoss-ssd.pdf => triton-quarterly/2015.03.30/2.1.6a-localstore-perf.pdf} (100%) create mode 100644 reports/triton-quarterly/2015.03.30/2.1.6a-localstore-perf.pdf.svg.emf create mode 100644 reports/triton-quarterly/2015.03.30/2.1.6a-localstore.txt Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.6a-localstore-perf.pdf.svg.emf b/reports/triton-quarterly/2015.03.30/2.1.6a-localstore-perf.pdf.svg.emf new file mode 100644 index 0000000..8712c36 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/2.1.6a-localstore-perf.pdf.svg.emf differ diff --git a/reports/triton-quarterly/2015.03.30/2.1.6a-localstore.txt b/reports/triton-quarterly/2015.03.30/2.1.6a-localstore.txt new file mode 100644 index 0000000..2dfcedd --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/2.1.6a-localstore.txt @@ -0,0 +1,40 @@ +75% complete [TODO: all of 2.1.6, not broken down into a/b/c] + +[TODO: run fio experiments as well.] + +In the previous quarterly report, we described the design of a local +storage abstraction layer developed collaboratively between Argonne +National Laboratory and Sandia National Laboratories. This abstraction +layer, now known as the Hierarchical Object Storage System (HOSS), +has been integrated into the Triton storage system prototype. +HOSS provides an object oriented interface to local storage with +versioning and indexing. The Triton/HOSS integration includes support +for two key Triton technologies: real-time replication and pipelining of +large data transfers. + +[2.1.6a-localstore-perf] + +We have also performed a preliminary evaluation of HOSS performance under +concurrent write workloads, as shown in Figure [TODO:ref]. This example uses +a microbenchmark that reflects expected server-attached storage workloads in +an HPC checkpoint use case. All experiments were performed on a Linux +3.16 workstation using the EXT4 file system and default tuning options. +A 240 GiB Intel 730 SSD was used as the underlying storage device. +The write access size was varied from 4 KiB to 4 MiB. All tests were +performed using direct I/O (to improve performance for high performance +storage devices), and each I/O operation was synchronized to disk before +being reported as complete (to evaluate durable write performance). + +Two HOSS examples are shown in [TODO:ref]: one that issued one operation +at a time, and one that issued up to 16 concurrent operations using Aesop +pbranches. Both examples provide record-granular versioning and indexing +that is not present in dd, a Linux file transfer utility that is shown +here for a baseline comparison. HOSS is optimized for the concurrent +access scenario, and the underlying Aesop concurrency model allows it +to saturate available storage bandwidth in that configuration despite +the additional features that it provides. As a result, it exceeds the +baseline dd performance by a wide margin at each access size when servicing a +concurrent workload. + +More in depth details about the local storage abstraction convergence can be +found in the complete report for this deliverable, see [TODO:ref]. hooks/post-receive --

1 0

branch, master, updated. 7e5bfcb3cee9e69803e0615ecb9f29528ef6f5f2
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 7e5bfcb3cee9e69803e0615ecb9f29528ef6f5f2 (commit) from defb97b794f5400d9dd531a727b18fc8534415b3 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 7e5bfcb3cee9e69803e0615ecb9f29528ef6f5f2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 12:39:12 2015 -0400 add draft text and figure for security deliverable ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/2.1.1-security-diagram.png | Bin 0 -> 102325 bytes .../triton-quarterly/2015.03.30/2.1.1-security.txt | 38 ++++++++++++++++++++ 2 files changed, 38 insertions(+), 0 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/2.1.1-security-diagram.png create mode 100644 reports/triton-quarterly/2015.03.30/2.1.1-security.txt Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.1-security-diagram.png b/reports/triton-quarterly/2015.03.30/2.1.1-security-diagram.png new file mode 100644 index 0000000..38bd1a9 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/2.1.1-security-diagram.png differ diff --git a/reports/triton-quarterly/2015.03.30/2.1.1-security.txt b/reports/triton-quarterly/2015.03.30/2.1.1-security.txt new file mode 100644 index 0000000..5ad97a7 --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/2.1.1-security.txt @@ -0,0 +1,38 @@ +60% complete + +[2.1.1-security-diagram.png] + +This work is being carried out in collaboration with Richard Brooks' team +at Clemson University and Lee Ward's team at Sandia National Laboratories. +Figure [TODO:ref] illustrates the basic protocol sequence diagram to be +used for authentication and authorization in the Triton storage system. +One of the key features of this architecture is that the storage +system itself is decoupled from the security infrastructure to the +extent possible in order to simplify the storage system and allow for +more flexibility. Clients will interact with authentication servers to +retrieve credentials and clients will interact with authorization servers +to retrieve capabilities. The object permission information used to +generate capabilities will be stored on the storage system itself, but +authorization servers will access this information on behalf of clients +using existing APIs. The primary role of a storage server in this model +is to verify capabilities, which it will do using a well defined API. + +The Clemson University team has produced a preliminary prototype which uses +the same network communication infrastructure as Triton (i.e., the Mercury +[TODO: cite] RPC framework) for communication. We have reviewed the security +APIs in this prototype and have begun work on integrating them into the +Triton storage system prototype. + +We are also continuing discussions with Clemson University and Sandia +National Laboratories to reach a consensus on key design elements in a way +that is beneficial to both the Triton and Sirocco storage systems. We have +reached agreement on the component model (as previously shown in Figure +[TODO:ref]), a communication abstraction method (so that other RPC systems +besides Mercury could be used if needed), and a revocation protocol (based on +the LWFS model [TODO:cite). We are continuing discussions on details such as +security granularity (should permissions apply just to objects or forks +within an object) and how exactly the permission metadata should be +represented in the ASG object storage data model. + +We hope to host one of Richard Brooks' students over the summer as a research +aide to work on finalizing the security prototype integration with Triton. hooks/post-receive --

1 0

Build failed in Jenkins: triton #203
by jenkins-ci＠mcs.anl.gov 17 Mar '15

17 Mar '15

See <https://jenkins-ci.mcs.anl.gov/job/triton/203/> ------------------------------------------ Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on trounce.mcs.anl.gov in workspace <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Deleting project workspace... done Checkout:triton / <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> - hudson.remoting.Channel@b1619a3:trounce.mcs.anl.gov Using strategy: Default Last Built Revision: Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Cloning the remote Git repository Cloning repository [email protected]:triton git --version git version 1.7.9.5 Fetching upstream changes from origin Commencing build of Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Checking out Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) [triton] $ /bin/sh -xe /tmp/hudson1473074985572615137.sh + code/scripts/jenkins/build-jenkins.sh code misc sim ======================================================================= = Build 203 2015-03-17_00-03-02 = on trounce.mcs.anl.gov [mcs trounce.mcs.anl.gov] ======================================================================= Triton source in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Building in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> ==================================================================== ===== INFO ========================================================= ==================================================================== Triton Source tree : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Workspace : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> Cachedir : /tmp/triton-cache ======================================================================= ==== STEP 1: Fetch build dependencies ================================= ======================================================================= openpa.tar.gz: (found in cache) OK (3ad998bb26ac84ee7de262db94dd7656) bmi.tar.gz: (found in cache) OK (82a8efe604a80c80d18dab76dfc13c38) boost.tar.bz2: (found in cache) OK (d6eef4b4cacb2183f2bf265a5a03a354) c-utils.tar.gz: (cached but invalid. Removing) (fetching) Build step 'Execute shell' marked build as failure TAP Reports Processing: START Looking for TAP results report in workspace using pattern: build/tritonbuild/tests/results.tap Did not find any matching files.

1 0

branch, master, updated. defb97b794f5400d9dd531a727b18fc8534415b3
by noreply＠mcs.anl.gov 16 Mar '15

16 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via defb97b794f5400d9dd531a727b18fc8534415b3 (commit) from ea7a9867c1bc9ff4e2678973033ac1ee88878ebe (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit defb97b794f5400d9dd531a727b18fc8534415b3 Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Mon Mar 16 11:26:58 2015 -0500 added ycsb qualitative assessment, cite ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/anl-quarterly-report-2015-03.docx | Bin 133920 -> 134717 bytes 1 files changed, 0 insertions(+), 0 deletions(-) Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx index 2e9d476..b6ee1b1 100644 Binary files a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx and b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx differ hooks/post-receive --

1 0

Jenkins build is back to normal : triton #202
by jenkins-ci＠mcs.anl.gov 16 Mar '15

16 Mar '15

See <https://jenkins-ci.mcs.anl.gov/job/triton/202/>

1 0

Build failed in Jenkins: triton #201
by jenkins-ci＠mcs.anl.gov 15 Mar '15

15 Mar '15

See <https://jenkins-ci.mcs.anl.gov/job/triton/201/> ------------------------------------------ Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on trounce.mcs.anl.gov in workspace <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Deleting project workspace... done Checkout:triton / <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> - hudson.remoting.Channel@2b65a035:trounce.mcs.anl.gov Using strategy: Default Last Built Revision: Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Cloning the remote Git repository Cloning repository [email protected]:triton git --version git version 1.7.9.5 Fetching upstream changes from origin Commencing build of Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Checking out Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) [triton] $ /bin/sh -xe /tmp/hudson5010303867522183592.sh + code/scripts/jenkins/build-jenkins.sh code misc sim ======================================================================= = Build 201 2015-03-15_00-02-05 = on trounce.mcs.anl.gov [mcs trounce.mcs.anl.gov] ======================================================================= Triton source in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Building in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> ==================================================================== ===== INFO ========================================================= ==================================================================== Triton Source tree : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Workspace : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> Cachedir : /tmp/triton-cache ======================================================================= ==== STEP 1: Fetch build dependencies ================================= ======================================================================= openpa.tar.gz: (found in cache) OK (3ad998bb26ac84ee7de262db94dd7656) bmi.tar.gz: (found in cache) OK (82a8efe604a80c80d18dab76dfc13c38) boost.tar.bz2: (found in cache) OK (d6eef4b4cacb2183f2bf265a5a03a354) c-utils.tar.gz: (cached but invalid. Removing) (fetching) Build step 'Execute shell' marked build as failure TAP Reports Processing: START Looking for TAP results report in workspace using pattern: build/tritonbuild/tests/results.tap Did not find any matching files.

1 0

Build failed in Jenkins: triton #200
by jenkins-ci＠mcs.anl.gov 14 Mar '15

14 Mar '15

See <https://jenkins-ci.mcs.anl.gov/job/triton/200/> ------------------------------------------ Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on trounce.mcs.anl.gov in workspace <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Deleting project workspace... done Checkout:triton / <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> - hudson.remoting.Channel@5789f322:trounce.mcs.anl.gov Using strategy: Default Last Built Revision: Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Cloning the remote Git repository Cloning repository [email protected]:triton git --version git version 1.7.9.5 Fetching upstream changes from origin Commencing build of Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Checking out Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) [triton] $ /bin/sh -xe /tmp/hudson8769593207390853927.sh + code/scripts/jenkins/build-jenkins.sh code misc sim ======================================================================= = Build 200 2015-03-14_00-03-02 = on trounce.mcs.anl.gov [mcs trounce.mcs.anl.gov] ======================================================================= Triton source in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Building in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> ==================================================================== ===== INFO ========================================================= ==================================================================== Triton Source tree : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Workspace : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> Cachedir : /tmp/triton-cache ======================================================================= ==== STEP 1: Fetch build dependencies ================================= ======================================================================= openpa.tar.gz: (found in cache) OK (3ad998bb26ac84ee7de262db94dd7656) bmi.tar.gz: (found in cache) OK (82a8efe604a80c80d18dab76dfc13c38) boost.tar.bz2: (found in cache) OK (d6eef4b4cacb2183f2bf265a5a03a354) c-utils.tar.gz: (cached but invalid. Removing) (fetching) Build step 'Execute shell' marked build as failure TAP Reports Processing: START Looking for TAP results report in workspace using pattern: build/tritonbuild/tests/results.tap Did not find any matching files.

1 0

branch, master, updated. ea7a9867c1bc9ff4e2678973033ac1ee88878ebe
by noreply＠mcs.anl.gov 13 Mar '15

13 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via ea7a9867c1bc9ff4e2678973033ac1ee88878ebe (commit) from 24378b556069be134662aadba95e5744d498358f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ea7a9867c1bc9ff4e2678973033ac1ee88878ebe Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Fri Mar 13 14:51:46 2015 -0500 short 2.1.4 deliverable text ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/anl-quarterly-report-2015-03.docx | Bin 133650 -> 133920 bytes 1 files changed, 0 insertions(+), 0 deletions(-) Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx index f14eb94..2e9d476 100644 Binary files a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx and b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx differ hooks/post-receive --

1 0

branch, master, updated. 24378b556069be134662aadba95e5744d498358f
by noreply＠mcs.anl.gov 13 Mar '15

13 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 24378b556069be134662aadba95e5744d498358f (commit) via c0e43177d5618a5ad435e3a032f95ddb6d89d649 (commit) from 2949b0fc92db750ab7a97ccc2571aa433e8ff4c2 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 24378b556069be134662aadba95e5744d498358f Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Fri Mar 13 14:40:58 2015 -0500 no need for scratch file now commit c0e43177d5618a5ad435e3a032f95ddb6d89d649 Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Fri Mar 13 14:39:13 2015 -0500 first cut at quaterly report, 2.1.3 deliverable ----------------------------------------------------------------------- Summary of changes: .../triton-quarterly/2015.03.30/2-1-3-figures.pptx | Bin 0 -> 33470 bytes .../triton-quarterly/2015.03.30/2.1.3-scratch.txt | 14 -------------- .../2015.03.30/anl-quarterly-report-2015-03.docx | Bin 22352 -> 133650 bytes 3 files changed, 0 insertions(+), 14 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/2-1-3-figures.pptx delete mode 100644 reports/triton-quarterly/2015.03.30/2.1.3-scratch.txt Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2-1-3-figures.pptx b/reports/triton-quarterly/2015.03.30/2-1-3-figures.pptx new file mode 100644 index 0000000..a6fa655 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/2-1-3-figures.pptx differ diff --git a/reports/triton-quarterly/2015.03.30/2.1.3-scratch.txt b/reports/triton-quarterly/2015.03.30/2.1.3-scratch.txt deleted file mode 100644 index f76a3a8..0000000 --- a/reports/triton-quarterly/2015.03.30/2.1.3-scratch.txt +++ /dev/null @@ -1,14 +0,0 @@ -Status: 75% complete (?) -Synopsis: - -In this quarter we turned focus to a complementary workload to IOR, YCSB (Yahoo Cloud Serving Benchmark) [1], which serves as platform for benchmarking common cloud computing scenarios. - -Description of YCSB -- grab some content from their paper - -Description of our integration with YCSB as a DB implementation -- java bindings for ASG via the SWIG tool -- initial ASG/Triton implementation in YCSB -- correctness testing ongoing - -In the following quarter, the ASG/Triton YCSB implementation will be refined, benchmarking will be performed on both the IOR and YCSB workloads, and analysis of both the benchmark results and mapping of semantics between the ASG/Triton object model and the respective benchmarks will be performed. The YCSB workloads will additionally be considered for use in the 2.1.4 deliverable. diff --git a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx index 5b6908e..f14eb94 100644 Binary files a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx and b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx differ hooks/post-receive --

1 0

Build failed in Jenkins: triton #199
by jenkins-ci＠mcs.anl.gov 13 Mar '15

13 Mar '15

See <https://jenkins-ci.mcs.anl.gov/job/triton/199/> ------------------------------------------ Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on trounce.mcs.anl.gov in workspace <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Deleting project workspace... done Checkout:triton / <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> - hudson.remoting.Channel@186825dd:trounce.mcs.anl.gov Using strategy: Default Last Built Revision: Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Cloning the remote Git repository Cloning repository [email protected]:triton git --version git version 1.7.9.5 Fetching upstream changes from origin Commencing build of Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) Checking out Revision 9c9d7b4da56fbac8bd3b1258198026ec6d0d91e2 (origin/master) [triton] $ /bin/sh -xe /tmp/hudson8715001194608922321.sh + code/scripts/jenkins/build-jenkins.sh code misc sim ======================================================================= = Build 199 2015-03-13_00-02-03 = on trounce.mcs.anl.gov [mcs trounce.mcs.anl.gov] ======================================================================= Triton source in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Building in <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> ==================================================================== ===== INFO ========================================================= ==================================================================== Triton Source tree : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/> Workspace : <https://jenkins-ci.mcs.anl.gov/job/triton/ws/build> Cachedir : /tmp/triton-cache ======================================================================= ==== STEP 1: Fetch build dependencies ================================= ======================================================================= openpa.tar.gz: (found in cache) OK (3ad998bb26ac84ee7de262db94dd7656) bmi.tar.gz: (found in cache) OK (82a8efe604a80c80d18dab76dfc13c38) boost.tar.bz2: (found in cache) OK (d6eef4b4cacb2183f2bf265a5a03a354) c-utils.tar.gz: (cached but invalid. Removing) (fetching) Build step 'Execute shell' marked build as failure TAP Reports Processing: START Looking for TAP results report in workspace using pattern: build/tritonbuild/tests/results.tap Did not find any matching files.

1 0