March 2015 - Triton-commits - mailman3.cels.anl.gov

branch, master, updated. e130299958e7fbe8bb606275c77899c779f30038
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via e130299958e7fbe8bb606275c77899c779f30038 (commit) from 43ce85cbe2bcf4d24577fe926881180d66225fc7 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e130299958e7fbe8bb606275c77899c779f30038 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 20:38:49 2015 -0400 update performance results ----------------------------------------------------------------------- Summary of changes: .../localstore-report.tex | 80 +++++++++---------- 1 files changed, 38 insertions(+), 42 deletions(-) Diff of changes: diff --git a/reports/triton-localstore-convergence/localstore-report.tex b/reports/triton-localstore-convergence/localstore-report.tex index 2370da5..b752020 100644 --- a/reports/triton-localstore-convergence/localstore-report.tex +++ b/reports/triton-localstore-convergence/localstore-report.tex @@ -200,22 +200,19 @@ stored in Triton. \subsection{Preliminary Evaluation} -\textcolor{red}{TODO: consider coming back and doing multithreaded baseline -measurements as well, maybe both with linear and random I/O?} - \begin{figure}[t] \centering - \subfigure[Recordstore (RS)]{ + \subfigure[16x concurrency]{ \centering - \includegraphics[width=0.45\textwidth]{figs/recordstore-ssd/recordstore-ssd.pdf} - \label{fig:rs-perf} + \includegraphics[width=0.45\textwidth]{figs/all-ssd/1way-ssd.pdf} + \label{fig:1w-perf} } - \subfigure[HOSS]{ + \subfigure[no concurrency]{ \centering - \includegraphics[width=0.45\textwidth]{figs/hoss-ssd/hoss-ssd.pdf} - \label{fig:hoss-perf} + \includegraphics[width=0.45\textwidth]{figs/all-ssd/16way-ssd.pdf} + \label{fig:16w-perf} } - \caption{Streaming write performance at the Recordstore and HOSS component level on an Intel 730 series SSD using direct I/O and + \caption{Streaming write performance on an Intel 730 series SSD using direct I/O and synchronizing each operation to disk.} \label{fig:perf} \end{figure} @@ -230,44 +227,43 @@ underlying storage device. The write access size was varied from 4 KiB to 4 MiB. All tests were performed using direct I/O (to improve performance for high performance storage devices), and each I/O operation was synchronized to disk before being reported as complete (to evaluate durable write -performance). +performance). We measured performance in two scenarios: one with no concurrency, using sequential +writes, and one with 16-way concurrency using sequentially issued writes. -The results of these experiments are shown in Figure~\ref{fig:perf}. -Linux \texttt{dd} command line utility performance is also shown for -comparison. \texttt{dd} was configured as follows: +As a baseline for comparison we used the \texttt{dd} and and \texttt{fio} benchmark utilities. The +\texttt{dd} utility is not capable of issuing concurrent I/O and thus is not shown in the 16-way +concurrency example. \texttt{dd} was configured as follows: \begin{verbatim} dd if=/dev/zero of=/tmp/1.dat bs=$SIZE count=50000 oflag=direct,dsync \end{verbatim} -Each configuration was executed and measured for 10 seconds. In the +\texttt{fio} was configured as follows: + +\begin{verbatim} +fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=$CONCURRENCY --direct=1 --thread \ +--offset_increment=$SIZE --filename=fio.dat --name=test --readwrite=write --blocksize=$SIZE \ +--runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 +\end{verbatim} + +Each benchmark was executed and measured for 10 seconds. In the case of \texttt{dd}, the transfer was halted after 10 seconds by sending a signal to the -benchmark process, while the Recordstore -benchmark automatically halted execution after 10 seconds. -All writes were issued sequentially starting from file offset 0 or record -index 0 (for \texttt{dd} file access and Recordstore object access, -respectively). - -Two Recordstore examples are shown in Figure~\ref{fig:rs-perf}: one that -issued one operation at a time, and one that issued up to 16 concurrent -operations using Aesop pbranches. Recordstore is optimized for the -latter scenario, and the underlying Aesop concurrency model allows it to -saturate available storage bandwidth. As a result, it exceeds the -baseline \texttt{dd} performance by a wide margin at each access size. -The non-concurrent workload, in contrast, does not match baseline -\texttt{dd} performance. We believe that performance for this workload could be -improved using \texttt{pthread} optimizations to reduce thread latency, -however. - -Figure~\ref{fig:hoss-perf} repeats the same experiment using the HOSS API -layer atop the Recordstore rather than using the Recordstore directly. -This API layer presents a full object interface with versioning and -indexing. It's performance trends largely match that of the direct -Recordstore write benchmark. The most noticeable discrepancy relative to -the Recordstore performance is in small (32 KiB or smaller) write access -without concurrency. Berkeley DB access latency is a significant factor -in response time for small writes, but this effect is mitigated by improved -utilization under concurrent workloads. +benchmark process, while the HOSS, Recordstore, and fio +benchmarks automatically halted execution after 10 seconds. + +The sequential results with no concurrency are shown in Figure~\ref{fig:1w-perf}. We see that +\texttt{dd} and \texttt{fio} perform similarly, reaching a peak of over 250 MiB/s. The recordstore +performance does not match the baseline performance due to multithreading overhead. Because it is +optimized for concurrent I/O activity, it always uses threads to perform I/O operations, while the +\texttt{dd} and \texttt{fio} benchmarks do not. The HOSS benchmark performs slightly slower than +Recrodstore by itself due to the additional latency of storing version and index metadata on each +operation. + +The sequential results with 16-way concurrency are shown +in Figure~\ref{fig:16w-perf}. Unlike the sequential case, the +Recordstore and HOSS optimizations and Aesop concurrency allow it +to match the baseline multithreaded \texttt{fio} performance despite +the additional record-granular versioning and indexing functionality. Recordstore and HOSS performance could be improved via thread tuning and Berkeley DB access optimization. Both components already @@ -275,7 +271,7 @@ perform well for highly concurrent workloads, however. We also note that we expect to achieve the same bandwidth for non-sequential access as well due to the log-structured nature of the Recordstore file module. We focused on sequential access patterns in these preliminary experiments for simplicity -and to facilitate comparison to the sequential \texttt{dd} utility. +and to facilitate comparison to the baseline \texttt{dd} and \texttt{fio} benchmarks. \section{Conclusions} hooks/post-receive --

1 0

branch, master, updated. 43ce85cbe2bcf4d24577fe926881180d66225fc7
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 43ce85cbe2bcf4d24577fe926881180d66225fc7 (commit) from e0439b51a081e3d3f349d421c27cbdd4532990e1 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 43ce85cbe2bcf4d24577fe926881180d66225fc7 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 20:24:05 2015 -0400 new experiments, including fio this time ----------------------------------------------------------------------- Summary of changes: .../figs/all-ssd/16way-ssd.pdf | Bin 0 -> 8075 bytes .../hoss-ssd.plt => all-ssd/16way-ssd.plt} | 8 +- .../figs/all-ssd/1way-ssd.pdf | Bin 0 -> 8493 bytes .../recordstore-ssd.plt => all-ssd/1way-ssd.plt} | 9 +- .../figs/all-ssd/all-sweep.sh | 80 ++++++++++++++++++++ .../figs/all-ssd/dd-1way.txt | 13 +++ .../figs/all-ssd/fio-16way.txt | 13 +++ .../figs/all-ssd/fio-1way.txt | 13 +++ .../figs/all-ssd/fio-example.txt | 1 + .../figs/all-ssd/hoss-16way.txt | 13 +++ .../figs/all-ssd/hoss-1way.txt | 13 +++ .../figs/{hoss-ssd => all-ssd}/hoss-ssd.plt | 0 .../figs/all-ssd/rs-16way.txt | 13 +++ .../figs/all-ssd/rs-1way.txt | 13 +++ 14 files changed, 181 insertions(+), 8 deletions(-) create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.pdf copy reports/triton-localstore-convergence/figs/{hoss-ssd/hoss-ssd.plt => all-ssd/16way-ssd.plt} (76%) create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.pdf copy reports/triton-localstore-convergence/figs/{recordstore-ssd/recordstore-ssd.plt => all-ssd/1way-ssd.plt} (77%) create mode 100755 reports/triton-localstore-convergence/figs/all-ssd/all-sweep.sh create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/dd-1way.txt create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/fio-16way.txt create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/fio-1way.txt create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/fio-example.txt create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/hoss-16way.txt create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/hoss-1way.txt copy reports/triton-localstore-convergence/figs/{hoss-ssd => all-ssd}/hoss-ssd.plt (100%) create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/rs-16way.txt create mode 100644 reports/triton-localstore-convergence/figs/all-ssd/rs-1way.txt Diff of changes: diff --git a/reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.pdf b/reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.pdf new file mode 100644 index 0000000..36308da Binary files /dev/null and b/reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.pdf differ diff --git a/reports/triton-localstore-convergence/figs/hoss-ssd/hoss-ssd.plt b/reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.plt similarity index 76% copy from reports/triton-localstore-convergence/figs/hoss-ssd/hoss-ssd.plt copy to reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.plt index 459edde..d626cb1 100644 --- a/reports/triton-localstore-convergence/figs/hoss-ssd/hoss-ssd.plt +++ b/reports/triton-localstore-convergence/figs/all-ssd/16way-ssd.plt @@ -9,7 +9,7 @@ set style line 4 lc 5 lw 2 set style line 5 lc 2 lw 2 set style increment user -set output "hoss-ssd.eps" +set output "16way-ssd.eps" set xlabel "Access size (bytes)" set ylabel "Bandwidth (MiB/s)" offset 1,0 @@ -24,7 +24,7 @@ set rmargin 5 set xtics ("4 KiB" 4096, "8 KiB" 8192, "16 KiB" 16384, "32 KiB" 32768, "64 KiB" 65536, "128 KiB" 131072, "256 KiB" 262144, "512 KiB" 524288, "1 MiB" 1048576, "2 MiB" 2097152, "4 MiB" 4194304) # set yrange [0:2100] -plot "hoss-16way.txt" using 1:2 with linespoints lt 1 pt 1 linecolor rgb "blue" title "HOSS (16x concurrency)", \ -"hoss-1way.txt" using 1:2 with linespoints lt 1 pt 2 linecolor rgb "green" title "HOSS (no concurrency)", \ -"dd-1way.txt" using 1:2 with linespoints lt 1 pt 3 linecolor rgb "orange" title "dd" +plot "hoss-16way.txt" using 1:2 with linespoints lt 1 pt 1 linecolor rgb "blue" title "HOSS", \ +"rs-16way.txt" using 1:2 with linespoints lt 1 pt 2 linecolor rgb "green" title "Recordstore", \ +"fio-16way.txt" using 1:2 with linespoints lt 1 pt 2 linecolor rgb "orange" title "fio" diff --git a/reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.pdf b/reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.pdf new file mode 100644 index 0000000..bf771ae Binary files /dev/null and b/reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.pdf differ diff --git a/reports/triton-localstore-convergence/figs/recordstore-ssd/recordstore-ssd.plt b/reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.plt similarity index 77% copy from reports/triton-localstore-convergence/figs/recordstore-ssd/recordstore-ssd.plt copy to reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.plt index 6c3d2d7..48ba22e 100644 --- a/reports/triton-localstore-convergence/figs/recordstore-ssd/recordstore-ssd.plt +++ b/reports/triton-localstore-convergence/figs/all-ssd/1way-ssd.plt @@ -9,7 +9,7 @@ set style line 4 lc 5 lw 2 set style line 5 lc 2 lw 2 set style increment user -set output "recordstore-ssd.eps" +set output "1way-ssd.eps" set xlabel "Access size (bytes)" set ylabel "Bandwidth (MiB/s)" offset 1,0 @@ -24,7 +24,8 @@ set rmargin 5 set xtics ("4 KiB" 4096, "8 KiB" 8192, "16 KiB" 16384, "32 KiB" 32768, "64 KiB" 65536, "128 KiB" 131072, "256 KiB" 262144, "512 KiB" 524288, "1 MiB" 1048576, "2 MiB" 2097152, "4 MiB" 4194304) # set yrange [0:2100] -plot "rs-16way.txt" using 1:2 with linespoints lt 1 pt 1 linecolor rgb "blue" title "RS (16x concurrency)", \ -"rs-1way.txt" using 1:2 with linespoints lt 1 pt 2 linecolor rgb "green" title "RS (no concurrency)", \ -"dd-1way.txt" using 1:2 with linespoints lt 1 pt 3 linecolor rgb "orange" title "dd" +plot "hoss-1way.txt" using 1:2 with linespoints lt 1 pt 1 linecolor rgb "blue" title "HOSS", \ +"rs-1way.txt" using 1:2 with linespoints lt 1 pt 2 linecolor rgb "green" title "Recordstore", \ +"fio-1way.txt" using 1:2 with linespoints lt 1 pt 2 linecolor rgb "orange" title "fio", \ +"dd-1way.txt" using 1:2 with linespoints lt 1 pt 3 linecolor rgb "purple" title "dd" diff --git a/reports/triton-localstore-convergence/figs/all-ssd/all-sweep.sh b/reports/triton-localstore-convergence/figs/all-ssd/all-sweep.sh new file mode 100755 index 0000000..0f318cf --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/all-sweep.sh @@ -0,0 +1,80 @@ +#!/bin/bash + +# simple script to sweep across access sizes doing write benchmarks. + +# all configurations use directio and synchronous operations + +echo "# ./recordstore-bench-write file alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore SIZE 16 10 foo.txt" > rs-16way.txt +echo -e "# <size>\t<MiB/s>" >> rs-16way.txt + +echo "# ./recordstore-bench-write file alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore SIZE 1 10 foo.txt" > rs-1way.txt +echo -e "# <size>\t<MiB/s>" >> rs-1way.txt + +echo "# dd if=/dev/zero of=/tmp/1.dat bs=SIZE count=50000 oflag=direct,dsync" > dd-1way.txt +echo -e "# <size>\t<MiB/s>" >> dd-1way.txt + +echo "# ./hoss-concurrent-io SIZE 16 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore" > hoss-16way.txt +echo -e "# <size>\t<MiB/s>" >> hoss-16way.txt + +echo "# ./hoss-concurrent-io SIZE 1 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore" > hoss-1way.txt +echo -e "# <size>\t<MiB/s>" >> hoss-1way.txt + +echo "# fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=16 --direct=1 --thread --offset_increment=4096 --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=4096 --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0" > fio-16way.txt +echo -e "# <size>\t<MiB/s>" >> fio-16way.txt + +echo "# fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=1 --direct=1 --thread --offset_increment=4096 --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=4096 --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0" > fio-1way.txt +echo -e "# <size>\t<MiB/s>" >> fio-1way.txt + +for i in 4096 8192 16384 32768 65536 131072 262144 524288 1048576 2097152 4194304; do + echo Running $i ... + + rm -rf /tmp/recordstore + sleep 5 + echo ./recordstore-bench-write file "alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore" $i 16 10 foo.txt + output=`./recordstore-bench-write file "alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore" $i 16 10 foo.txt | grep MiB |cut -d ' ' -f 2` + echo -e "$i\t$output" >> rs-16way.txt + + rm -rf /tmp/recordstore + sleep 5 + echo ./recordstore-bench-write file "alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore" $i 1 10 foo.txt + output=`./recordstore-bench-write file "alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore" $i 1 10 foo.txt |grep MiB | cut -d ' ' -f 2` + echo -e "$i\t$output" >> rs-1way.txt + + rm -rf /tmp/recordstore + sleep 5 + echo ./hoss-concurrent-io $i 16 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore + output=`./hoss-concurrent-io $i 16 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore | grep MiB |cut -d ' ' -f 2` + echo -e "$i\t$output" >> hoss-16way.txt + + rm -rf /tmp/recordstore + sleep 5 + echo ./hoss-concurrent-io $i 1 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore + output=`./hoss-concurrent-io $i 1 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore | grep MiB |cut -d ' ' -f 2` + echo -e "$i\t$output" >> hoss-1way.txt + + + rm -rf /tmp/dd.dat + sleep 5 + echo dd if=/dev/zero of=/tmp/1.dat bs=$i count=50000 oflag=direct,dsync + output=`timeout -s USR1 -k 11 10 dd if=/dev/zero of=/tmp/1.dat bs=$i count=50000 oflag=direct,dsync 2>&1 |grep bytes |head -n 1| cut -d ' ' -f 1` + output=`echo $output/1024^2/10 | bc -l` + echo -e "$i\t$output" >> dd-1way.txt + + rm -rf /tmp/fio.dat + sleep 5 + echo fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=16 --direct=1 --thread --offset_increment=$i --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=$i --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 + output=`fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=16 --direct=1 --thread --offset_increment=$i --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=$i --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 |grep WRITE | cut -d = -f 3 | cut -d K -f 1` + output=`echo $output/1024 | bc -l` + echo -e "$i\t$output" >> fio-16way.txt + + rm -rf /tmp/fio.dat + sleep 5 + echo fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=1 --direct=1 --thread --offset_increment=$i --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=$i --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 + output=`fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=1 --direct=1 --thread --offset_increment=$i --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=$i --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 |grep WRITE | cut -d = -f 3 | cut -d K -f 1` + output=`echo $output/1024 | bc -l` + echo -e "$i\t$output" >> fio-1way.txt + + +done + + diff --git a/reports/triton-localstore-convergence/figs/all-ssd/dd-1way.txt b/reports/triton-localstore-convergence/figs/all-ssd/dd-1way.txt new file mode 100644 index 0000000..e17dd4f --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/dd-1way.txt @@ -0,0 +1,13 @@ +# dd if=/dev/zero of=/tmp/1.dat bs=SIZE count=50000 oflag=direct,dsync +# <size> <MiB/s> +4096 7.69570312500000000000 +8192 15.19843750000000000000 +16384 28.91562500000000000000 +32768 53.44375000000000000000 +65536 92.86250000000000000000 +131072 148.85000000000000000000 +262144 204.55000000000000000000 +524288 248.20000000000000000000 +1048576 225.60000000000000000000 +2097152 263.80000000000000000000 +4194304 261.60000000000000000000 diff --git a/reports/triton-localstore-convergence/figs/all-ssd/fio-16way.txt b/reports/triton-localstore-convergence/figs/all-ssd/fio-16way.txt new file mode 100644 index 0000000..e9ce62a --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/fio-16way.txt @@ -0,0 +1,13 @@ +# fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=16 --direct=1 --thread --offset_increment=4096 --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=4096 --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 +# <size> <MiB/s> +4096 24.14453125000000000000 +8192 45.18066406250000000000 +16384 76.33398437500000000000 +32768 113.63964843750000000000 +65536 145.15625000000000000000 +131072 240.55664062500000000000 +262144 258.03515625000000000000 +524288 283.59082031250000000000 +1048576 234.55175781250000000000 +2097152 240.79882812500000000000 +4194304 265.57617187500000000000 diff --git a/reports/triton-localstore-convergence/figs/all-ssd/fio-1way.txt b/reports/triton-localstore-convergence/figs/all-ssd/fio-1way.txt new file mode 100644 index 0000000..7da6d76 --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/fio-1way.txt @@ -0,0 +1,13 @@ +# fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=1 --direct=1 --thread --offset_increment=4096 --filename=/tmp/fio.dat --name=test --readwrite=write --blocksize=4096 --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 +# <size> <MiB/s> +4096 8.38867187500000000000 +8192 16.26074218750000000000 +16384 31.63671875000000000000 +32768 57.53710937500000000000 +65536 102.10156250000000000000 +131072 161.23339843750000000000 +262144 218.75292968750000000000 +524288 244.62500000000000000000 +1048576 212.35742187500000000000 +2097152 273.37207031250000000000 +4194304 226.85937500000000000000 diff --git a/reports/triton-localstore-convergence/figs/all-ssd/fio-example.txt b/reports/triton-localstore-convergence/figs/all-ssd/fio-example.txt new file mode 100644 index 0000000..a9464ed --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/fio-example.txt @@ -0,0 +1 @@ +fio --ioengine=psync --iodepth=1 --group_reporting --numjobs=16 --direct=1 --thread --offset_increment=4096 --filename=fio.dat --name=test --readwrite=write --blocksize=4096 --runtime=10 --fdatasync=1 --unlink=1 --size=4G --overwrite=0 diff --git a/reports/triton-localstore-convergence/figs/all-ssd/hoss-16way.txt b/reports/triton-localstore-convergence/figs/all-ssd/hoss-16way.txt new file mode 100644 index 0000000..933f4ad --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/hoss-16way.txt @@ -0,0 +1,13 @@ +# ./hoss-concurrent-io SIZE 16 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore +# <size> <MiB/s> +4096 17.842281 +8192 38.173045 +16384 74.306832 +32768 134.129277 +65536 217.901779 +131072 247.715711 +262144 222.164420 +524288 270.453487 +1048576 250.782106 +2097152 267.427411 +4194304 259.783157 diff --git a/reports/triton-localstore-convergence/figs/all-ssd/hoss-1way.txt b/reports/triton-localstore-convergence/figs/all-ssd/hoss-1way.txt new file mode 100644 index 0000000..3cc6b7e --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/hoss-1way.txt @@ -0,0 +1,13 @@ +# ./hoss-concurrent-io SIZE 1 10 alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore +# <size> <MiB/s> +4096 4.741457 +8192 9.505135 +16384 18.453399 +32768 33.820008 +65536 62.409263 +131072 92.582991 +262144 117.070096 +524288 164.415751 +1048576 194.458195 +2097152 202.233507 +4194304 205.224570 diff --git a/reports/triton-localstore-convergence/figs/hoss-ssd/hoss-ssd.plt b/reports/triton-localstore-convergence/figs/all-ssd/hoss-ssd.plt similarity index 100% copy from reports/triton-localstore-convergence/figs/hoss-ssd/hoss-ssd.plt copy to reports/triton-localstore-convergence/figs/all-ssd/hoss-ssd.plt diff --git a/reports/triton-localstore-convergence/figs/all-ssd/rs-16way.txt b/reports/triton-localstore-convergence/figs/all-ssd/rs-16way.txt new file mode 100644 index 0000000..943a8db --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/rs-16way.txt @@ -0,0 +1,13 @@ +# ./recordstore-bench-write file alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore SIZE 16 10 foo.txt +# <size> <MiB/s> +4096 31.201774 +8192 50.826861 +16384 90.659295 +32768 134.996636 +65536 203.970100 +131072 249.341018 +262144 264.029877 +524288 275.844507 +1048576 267.460571 +2097152 253.057567 +4194304 279.691897 diff --git a/reports/triton-localstore-convergence/figs/all-ssd/rs-1way.txt b/reports/triton-localstore-convergence/figs/all-ssd/rs-1way.txt new file mode 100644 index 0000000..5be78c7 --- /dev/null +++ b/reports/triton-localstore-convergence/figs/all-ssd/rs-1way.txt @@ -0,0 +1,13 @@ +# ./recordstore-bench-write file alignment:4096,O_SYNC,O_DIRECT,sync_highwater:8,path:/tmp/recordstore SIZE 1 10 foo.txt +# <size> <MiB/s> +4096 7.733682 +8192 13.299131 +16384 26.332521 +32768 48.480715 +65536 83.783139 +131072 116.503291 +262144 141.450308 +524288 187.664255 +1048576 209.231681 +2097152 212.167373 +4194304 226.300856 hooks/post-receive --

1 0

branch, master, updated. e0439b51a081e3d3f349d421c27cbdd4532990e1
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via e0439b51a081e3d3f349d421c27cbdd4532990e1 (commit) via a6a318261d6587716e22bc10a92d85443fa6c7b4 (commit) from 3a9a44318a0018bc8f37b40b3ecdc8bb15561012 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e0439b51a081e3d3f349d421c27cbdd4532990e1 Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Tue Mar 17 16:17:32 2015 -0500 put software text into quarterly report commit a6a318261d6587716e22bc10a92d85443fa6c7b4 Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Tue Mar 17 15:50:32 2015 -0500 put security text into word doc ----------------------------------------------------------------------- Summary of changes: .../triton-quarterly/2015.03.30/2.1.1-security.txt | 38 -------------------- .../2015.03.30/anl-quarterly-report-2015-03.docx | Bin 134717 -> 239748 bytes reports/triton-quarterly/2015.03.30/software.txt | 27 -------------- 3 files changed, 0 insertions(+), 65 deletions(-) delete mode 100644 reports/triton-quarterly/2015.03.30/2.1.1-security.txt delete mode 100644 reports/triton-quarterly/2015.03.30/software.txt Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.1-security.txt b/reports/triton-quarterly/2015.03.30/2.1.1-security.txt deleted file mode 100644 index 5ad97a7..0000000 --- a/reports/triton-quarterly/2015.03.30/2.1.1-security.txt +++ /dev/null @@ -1,38 +0,0 @@ -60% complete - -[2.1.1-security-diagram.png] - -This work is being carried out in collaboration with Richard Brooks' team -at Clemson University and Lee Ward's team at Sandia National Laboratories. -Figure [TODO:ref] illustrates the basic protocol sequence diagram to be -used for authentication and authorization in the Triton storage system. -One of the key features of this architecture is that the storage -system itself is decoupled from the security infrastructure to the -extent possible in order to simplify the storage system and allow for -more flexibility. Clients will interact with authentication servers to -retrieve credentials and clients will interact with authorization servers -to retrieve capabilities. The object permission information used to -generate capabilities will be stored on the storage system itself, but -authorization servers will access this information on behalf of clients -using existing APIs. The primary role of a storage server in this model -is to verify capabilities, which it will do using a well defined API. - -The Clemson University team has produced a preliminary prototype which uses -the same network communication infrastructure as Triton (i.e., the Mercury -[TODO: cite] RPC framework) for communication. We have reviewed the security -APIs in this prototype and have begun work on integrating them into the -Triton storage system prototype. - -We are also continuing discussions with Clemson University and Sandia -National Laboratories to reach a consensus on key design elements in a way -that is beneficial to both the Triton and Sirocco storage systems. We have -reached agreement on the component model (as previously shown in Figure -[TODO:ref]), a communication abstraction method (so that other RPC systems -besides Mercury could be used if needed), and a revocation protocol (based on -the LWFS model [TODO:cite). We are continuing discussions on details such as -security granularity (should permissions apply just to objects or forks -within an object) and how exactly the permission metadata should be -represented in the ASG object storage data model. - -We hope to host one of Richard Brooks' students over the summer as a research -aide to work on finalizing the security prototype integration with Triton. diff --git a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx index b6ee1b1..e5aa4d6 100644 Binary files a/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx and b/reports/triton-quarterly/2015.03.30/anl-quarterly-report-2015-03.docx differ diff --git a/reports/triton-quarterly/2015.03.30/software.txt b/reports/triton-quarterly/2015.03.30/software.txt deleted file mode 100644 index 2b28edf..0000000 --- a/reports/triton-quarterly/2015.03.30/software.txt +++ /dev/null @@ -1,27 +0,0 @@ -Based on feedback from previous quarterly reports we will begin including -source code snapshots with our reports. In this quarterly report we -feature two component packages with complete installation documentation: - -Aesop - -Aesop was developed primarily by ANL. It is a programming language -(based on c), programming model, and runtime system designed to facilitate -the construction of highly concurrent system software. It serves as the -basis for both the Triton (ANL) and and Sirocco (Sandia) storage systems. - -The source code for Aesop can be found in [TODO:ref] and the installation -instructions can be found in [TODO:ref]. - -Mercury - -Mercury was developed as a collaboration between the HDF Group and ANL. It -is remote procedure call (RPC) framework designed for use on high performance -computing systems. One if its most notable features is a mechanism for -efficient bulk data transfers in the context of RPC operations. -Mercury supports a variety of underlying network transports, including SSM, -BMI, CCI, and MPI. It serves as the communication management layer for the -Triton (ANL) storage system. - -The source code for Mercury can be found in [TODO:ref] and the installation -instructions can be found in [TODO:ref]. - hooks/post-receive --

1 0

branch, master, updated. 3a9a44318a0018bc8f37b40b3ecdc8bb15561012
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 3a9a44318a0018bc8f37b40b3ecdc8bb15561012 (commit) from 44b504be9801bb87b4c26c8b5925e2d08b79edfd (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3a9a44318a0018bc8f37b40b3ecdc8bb15561012 Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Tue Mar 17 15:44:50 2015 -0500 autonomous object reorg text ----------------------------------------------------------------------- Summary of changes: .../triton-quarterly/2015.03.30/2.1.5-reorg.txt | 21 ++++++++++++++++++++ 1 files changed, 21 insertions(+), 0 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/2.1.5-reorg.txt Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.5-reorg.txt b/reports/triton-quarterly/2015.03.30/2.1.5-reorg.txt new file mode 100644 index 0000000..7017c7d --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/2.1.5-reorg.txt @@ -0,0 +1,21 @@ +Status: 25% +Synopsis: + +In this quarter, we have begun an investigation into reorganization of objects to reflect system and application workloads. Our initial exploration has been informed by the trends towards deeper memory/storage hierarchies, as evidenced by numerous projects investigating the usage of nonvolatile burst buffers [burst buffers,ff], recent storage medium advancements [2], and purpose-driven storage technologies specializing storage by data age and "temperature" (rate of access) [f4,pelican]. For such systems, there are numerous significant design questions to address. For example, should the namespace of these layers be unified under a single naming service or disjoint and managed by separate services? What are the costs of moving data between levels of the hierarchy, and under what conditions does it make sense to perform such moves? + +With respect to Triton, our plan is to first investigate the design points for Triton within such an architecture. Second, we plan to zoom out and investigate how different design aspects in Triton can inform hierarchical system architecture. + +As an example of the former, we plan to examine the feasibility of Triton as both a "hot" and "warm" storage layer, to use the "temperature" connotations in recent literature. In other words, we envision Triton serving as a persistent storage layer underneath burst storage media, for data that is both frequently accessed and accessed at a rate that would be unacceptable for more capacity-optimized storage layers. In this context, it is typically preferable for "hot" data to be replicated (resulting in a high redundancy ratio) to avoid erasure-coding overheads, while "warm" data is accessed at such a rate that erasure coding overheads (resulting in a lower redundancy ratio) are acceptable for access performance. In general, managing "hot" and "warm" data, especially in an automated manner, requires a good treatment of cost models and algorithms data transition between "hot" and "warm" storage modes (a frequently cited example being HP's AutoRAID system [TODO:autoraid]). Speci fic to Triton, there are a few challenges and research questions to explore: +1) how do we efficiently serve I/O requests to parity-coded objects distributed across multiple servers, while keeping with our design goals of server-directed I/O and conditional update functionality? +2) how does the use of parity-coded objects affect our local storage strategy, which is currently log-structured? +3) how do we consistently map RAID stripes in the face of failure while avoiding full object re-coding? + +(TODO: need to iterate on this more) +As an example of the latter, we plan to investigate the use of Triton's container/object/fork/record data model in a hierarchical storage context. In addition, the combination of consistent object location algorithms with tier-aware location services could be an area of study. + +[??] TODO: burst buffer cite. Fast Forward? +[??] TODO: fast forward project? +[??] TODO: shingle drive +[?] Subramanian Muralidhar et al., "F4: Facebook’s Warm BLOB Storage System," in Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, Berkeley, CA, USA, 2014, pp. 383–398. +[?] Shobana Balakrishnan et al., "Pelican: A Building Block for Exascale Cold Data Storage," in Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation, Berkeley, CA, USA, 2014, pp. 351–365. +[?] John Wilkes et al., "The HP AutoRAID Hierarchical Storage System," ACM Trans. Comput. Syst., vol. 14, no. 1, pp. 108–136, Feb. 1996. hooks/post-receive --

1 0

branch, master, updated. 44b504be9801bb87b4c26c8b5925e2d08b79edfd
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 44b504be9801bb87b4c26c8b5925e2d08b79edfd (commit) from e2d930c65f35fa092c3972326745bf27b9283836 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 44b504be9801bb87b4c26c8b5925e2d08b79edfd Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 15:21:08 2015 -0400 add rough text for produced software ----------------------------------------------------------------------- Summary of changes: reports/triton-quarterly/2015.03.30/software.txt | 27 ++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/software.txt Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/software.txt b/reports/triton-quarterly/2015.03.30/software.txt new file mode 100644 index 0000000..2b28edf --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/software.txt @@ -0,0 +1,27 @@ +Based on feedback from previous quarterly reports we will begin including +source code snapshots with our reports. In this quarterly report we +feature two component packages with complete installation documentation: + +Aesop + +Aesop was developed primarily by ANL. It is a programming language +(based on c), programming model, and runtime system designed to facilitate +the construction of highly concurrent system software. It serves as the +basis for both the Triton (ANL) and and Sirocco (Sandia) storage systems. + +The source code for Aesop can be found in [TODO:ref] and the installation +instructions can be found in [TODO:ref]. + +Mercury + +Mercury was developed as a collaboration between the HDF Group and ANL. It +is remote procedure call (RPC) framework designed for use on high performance +computing systems. One if its most notable features is a mechanism for +efficient bulk data transfers in the context of RPC operations. +Mercury supports a variety of underlying network transports, including SSM, +BMI, CCI, and MPI. It serves as the communication management layer for the +Triton (ANL) storage system. + +The source code for Mercury can be found in [TODO:ref] and the installation +instructions can be found in [TODO:ref]. + hooks/post-receive --

1 0

branch, master, updated. e2d930c65f35fa092c3972326745bf27b9283836
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via e2d930c65f35fa092c3972326745bf27b9283836 (commit) from a5ca6ae030dc8fb3477f309cca6e1c81bf21d28e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e2d930c65f35fa092c3972326745bf27b9283836 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 15:11:14 2015 -0400 source and doc snapshots to include with report ----------------------------------------------------------------------- Summary of changes: .../source-code/ae-blocking-parser-0.2.tar.gz | Bin 0 -> 124035 bytes .../2015.03.30/source-code/aesop-0.2.1-install.pdf | Bin 0 -> 78556 bytes .../2015.03.30/source-code/aesop-0.2.1.tar.gz | Bin 0 -> 264069 bytes .../source-code/mercury-0.8.2-readme.txt | 148 ++++++++++++++++++++ .../2015.03.30/source-code/mercury-0.8.2.tar.bz2 | Bin 0 -> 327266 bytes 5 files changed, 148 insertions(+), 0 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/source-code/ae-blocking-parser-0.2.tar.gz create mode 100644 reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1-install.pdf create mode 100644 reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1.tar.gz create mode 100644 reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2-readme.txt create mode 100644 reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2.tar.bz2 Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/source-code/ae-blocking-parser-0.2.tar.gz b/reports/triton-quarterly/2015.03.30/source-code/ae-blocking-parser-0.2.tar.gz new file mode 100644 index 0000000..baf7782 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/source-code/ae-blocking-parser-0.2.tar.gz differ diff --git a/reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1-install.pdf b/reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1-install.pdf new file mode 100644 index 0000000..25d00d4 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1-install.pdf differ diff --git a/reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1.tar.gz b/reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1.tar.gz new file mode 100644 index 0000000..29d3524 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/source-code/aesop-0.2.1.tar.gz differ diff --git a/reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2-readme.txt b/reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2-readme.txt new file mode 100644 index 0000000..6074292 --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2-readme.txt @@ -0,0 +1,148 @@ +MERCURY +------- + +These are the release notes for Mercury (as of 11/19/2014). + +WHAT IS MERCURY? + + Mercury is an RPC framework specifically designed for use in HPC systems + that allows asynchronous transfer of parameters and execution requests + and direct support of large data arguments. The interface is generic + to allow any function call to be shipped. The network implementation + is abstracted, allowing easy porting to future systems and efficient use + of existing native transport mechanisms. + + Please see the accompanying COPYING file for license details. + + Contributions and patches are welcomed but require a Contributor License + Agreement (CLA) to be filled out. Please contact us if you are interested + in contributing to Mercury. + +ARCHITECTURES SUPPORTED: + + Architectures supported by MPI implementations are supported by the + network abstraction layer. Both MPI and BMI plugins fully + implement the network abstraction layer and are currently supported. + Additional plugins are currently in development and will be added in + future releases to support additional network protocols. + +DOCUMENTATION: + + - Please see the accompanying design implementation documentation for + additional and useful details. + +SOFTWARE REQUIREMENTS: + + Compiling and running Mercury requires up-to-date versions of various + software packages. Beware that using excessively old versions of these + packages can cause indirect errors that are very difficult to track down. + To make use of the MPI plugin, Mercury requires a _well-configured_ MPI + implementation (MPICH2 v1.4.1 or higher / OpenMPI v1.6 or higher). For + optional automatic code generation features (which are used for generating + encoding and decoding routines), the preprocessor subset of the BOOST + library must be included (Boost v1.48 or higher is recommended). On Linux + OpenPA v1.0.3 or higher is required (the version that is included with MPICH + can also be used). + +BUILDING: + + - If you install the full sources, put the tarball in a directory where you + have permissions (e.g., your home directory) and unpack it: + + gzip -cd mercury-X.tar.gz | tar xvf - + + or + + bzip2 -dc mercury-X.tar.bz2 | tar xvf - + + Replace "X" with the version number of the package. + + - (Optional) If you checked out the sources using git and want to build + the testing suite (which requires the kwsys submodule), you need to issue + from the root of the source directory the following command: + + git submodule update --init + + - Mercury makes use of the CMake build-system and requires that you do an + out-of-source build. In order to do that, you must create a new build + directory and run the 'ccmake' command from it: + + cd mercury-X + mkdir build + cd build + ccmake .. (where ".." is the relative path to the mercury-X directory) + + - Type 'c' multiple times and choose suitable options. Recommended options + are: + + BUILD_SHARED_LIBS ON (or OFF if the library you link + against requires static libraries) + BUILD_TESTING ON + Boost_INCLUDE_DIR /path/to/include/directory + CMAKE_INSTALL_PREFIX /path/to/install/directory + MERCURY_ENABLE_PARALLEL_TESTING ON + MERCURY_USE_BOOST_PP ON + MERCURY_USE_SYSTEM_MCHECKSUM OFF + MERCURY_USE_XDR OFF + NA_USE_MPI ON + OPA_INCLUDE_DIR /path/to/include/directory + + Setting include directory and library paths may require you to toggle to + the advanced mode by typing 't'. Once you are done and do not see any + errors, type 'g' to generate makefiles. Once you exit the CMake + configuration screen and are ready to build the targets, do: + + make + + - (Optional) Verbose compile/build output: + + This is done by inserting "VERBOSE=1" in the "make" command. E.g.: + + make VERBOSE=1 + +INSTALLING: + + - Assuming that the CMAKE_INSTALL_PREFIX has been set (see previous step) + and that you have write permissions to the destination directory, do + from the build directory: + + make install + +TESTING: + + - Tests can be run to check that basic function shipping (metadata and bulk + data transfers) is properly working. CTest is used to run the tests, + simply run from the build directory: + + ctest . + + - (Optional) Verbose testing: + + This is done by inserting "-V" in the "ctest" command. E.g.: + + ctest -V . + + Extra verbose information can be displayed by inserting "-VV". E.g.: + + ctest -VV . + + - Tests run with one server process and X client processes. To change the + number of client processes that are being used, the MPIEXEC_MAX_NUMPROCS + variable needs to be modified (toggle to advanced mode if you do not see + it). The default value is 2. + Note that you need to run 'make' again after the makefile generation + to use the new value. Note also that this variable needs to be changed + if you run the tests manually and use a different number of client + processes. + + - (Optional) To run the tests manually with the MPI plugin open up two + terminal windows, one for the server and one for the client. From the same + directory where you have write permissions (so that the port configuration + file can be written by the server and read by the client) do: + + mpirun -np 1 /path/to/binary/hg_test_server -c mpi + + and in the other: + + mpirun -np 2 /path/to/binary/hg_test_TESTNAME -c mpi + diff --git a/reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2.tar.bz2 b/reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2.tar.bz2 new file mode 100644 index 0000000..efa6066 Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/source-code/mercury-0.8.2.tar.bz2 differ hooks/post-receive --

1 0

branch, master, updated. a5ca6ae030dc8fb3477f309cca6e1c81bf21d28e
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via a5ca6ae030dc8fb3477f309cca6e1c81bf21d28e (commit) from bd97689174a067aaad1a234dd90f6d37334f7fe8 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit a5ca6ae030dc8fb3477f309cca6e1c81bf21d28e Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Tue Mar 17 13:56:58 2015 -0500 some fault detection text (whoops, redundant work) ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/2.1.6c-fault-detection.txt | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt b/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt index 55bd3c7..f9ced58 100644 --- a/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt +++ b/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt @@ -9,3 +9,6 @@ opportunity for overlap, but we are working to clarify how differences in server visibility and the client's role in the storage system impact the fault detection model. +=== John's overly long text + +Additionally, we are engaged in ongoing discussions with the Sandia team on how these algorithms can adapt under different system architectures. In particular, the fault detection / dissemination strategy in Triton is built upon the assumption that there is a weakly-consistent view of the entire storage cluster. However, in Sirocco, no one storage server has a complete (consistent or otherwise) view of the cluster. In other words, Sirocco is built upon an overlay network abstraction in which objects can be located via a distributed routing algorithm. We are in the process of determining what implications this has on the fault detection/dissemination design. For example, the SWIM algorithm described in previous reports cannot directly be used on a network in which not all entities are known at each sender-side. However, properties of Sirocco's overlay network may allow SWIM or another algorithm to be adapted for use by communicating only with subsets of members in the overall storage cluster. In the next quarter we plan on reconciling these differences. hooks/post-receive --

1 0

branch, master, updated. bd97689174a067aaad1a234dd90f6d37334f7fe8
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via bd97689174a067aaad1a234dd90f6d37334f7fe8 (commit) via 1f0740dd87b77a499351ed1bc5564ba214549563 (commit) from beaa00dc98b3d3f3fb35d631ae580d603aaf914e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit bd97689174a067aaad1a234dd90f6d37334f7fe8 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 14:43:59 2015 -0400 rough text for 2.1.6c fault detection commit 1f0740dd87b77a499351ed1bc5564ba214549563 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Mar 17 14:30:27 2015 -0400 minor wording tweak ----------------------------------------------------------------------- Summary of changes: .../localstore-report.tex | 4 ++-- .../2015.03.30/2.1.6c-fault-detection.txt | 11 +++++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt Diff of changes: diff --git a/reports/triton-localstore-convergence/localstore-report.tex b/reports/triton-localstore-convergence/localstore-report.tex index f2b61b7..2370da5 100644 --- a/reports/triton-localstore-convergence/localstore-report.tex +++ b/reports/triton-localstore-convergence/localstore-report.tex @@ -20,9 +20,9 @@ for managing client/server communication, and convergence on an overall fault detection strategy.'' Specifically, it addresses the first portion of this deliverable: convergence on a shared API for local storage abstraction. -The Triton (ANL) and Sirocco (Sandia) teams each developed local storage +In previous work, the Triton (ANL) and Sirocco (Sandia) teams each developed local storage abstraction prototypes to serve as proofs of concept and to explore local -storage semantics in previous work. We identified common requirements +storage semantics. These early prototypes were used to identify common requirements for variable-sized, record-oriented access models, atomic primitives, and record-granular versioning. We then leveraged lessons learned from the initial prototypes to produce a new, common local storage component diff --git a/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt b/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt new file mode 100644 index 0000000..55bd3c7 --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/2.1.6c-fault-detection.txt @@ -0,0 +1,11 @@ +In our previous quarterly report, we outlined a methodology for +exploring the design space of both epidemic and semi-centralized fault +detection protocols using discrete event simulation. These activities +are ongoing. + +We are also working with the Sirocco (Sandia) team to evaluate commonalities +in our respective fault detection requirements. There is significant +opportunity for overlap, but we are working to clarify how differences in server +visibility and the client's role in the storage system impact the fault +detection model. + hooks/post-receive --

1 0

branch, master, updated. beaa00dc98b3d3f3fb35d631ae580d603aaf914e
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via beaa00dc98b3d3f3fb35d631ae580d603aaf914e (commit) from c96458a3fde257a9c2769f4aca1f9f3b06fb8af5 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit beaa00dc98b3d3f3fb35d631ae580d603aaf914e Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Tue Mar 17 13:33:57 2015 -0500 prov citations ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/2.1.2-provenance.txt | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt b/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt index f82699b..84eb68b 100644 --- a/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt +++ b/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt @@ -19,6 +19,6 @@ We have made the following progress on these challenges, informing our prototype In the next quarter, we plan to finish the related studies and turn focus to integrating with Triton, either as a standalone service coordinating with Triton (metadata server as-a-client) or as a protocol within Triton. -[1] TODO: metadata graph pub -[2] TODO: IndexFS -[3] TODO: BatchFS +[1] Dong Dai, Robert B. Ross, Philip Carns, Dries Kimpe, and Yong Chen, "Using Property Graphs for Rich Metadata Management in HPC Systems," in Proceedings of the 9th Parallel Data Storage Workshop, Piscataway, NJ, USA, 2014, pp. 7–12. +[2] Kai Ren, Qing Zheng, Swapnil Patil, and Garth Gibson, "IndexFS: Scaling File System Metadata Performance with Stateless Caching and Bulk Insertion," in High Performance Computing, Networking, Storage and Analysis, SC14: International Conference for, 2014, pp. 237–248. +[3] Qing Zheng, Kai Ren, and Garth Gibson, "BatchFS: Scaling the File System Control Plane with Client-Funded Metadata Servers," in Parallel Data Storage Workshop (PDSW), 2014 9th, 2014, pp. 1–6. hooks/post-receive --

1 0

branch, master, updated. c96458a3fde257a9c2769f4aca1f9f3b06fb8af5
by noreply＠mcs.anl.gov 17 Mar '15

17 Mar '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via c96458a3fde257a9c2769f4aca1f9f3b06fb8af5 (commit) from fd43fbe54cacc9394d30a35c289d37e8753b2ff2 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit c96458a3fde257a9c2769f4aca1f9f3b06fb8af5 Author: John Jenkins <jenkins(a)mcs.anl.gov> Date: Tue Mar 17 13:28:54 2015 -0500 initial provenance report text ----------------------------------------------------------------------- Summary of changes: .../2015.03.30/2.1.2-provenance.txt | 24 ++++++++++++++++++++ .../2015.03.30/asynctravel-2_cropped.jpeg | Bin 0 -> 370408 bytes 2 files changed, 24 insertions(+), 0 deletions(-) create mode 100644 reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt create mode 100644 reports/triton-quarterly/2015.03.30/asynctravel-2_cropped.jpeg Diff of changes: diff --git a/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt b/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt new file mode 100644 index 0000000..f82699b --- /dev/null +++ b/reports/triton-quarterly/2015.03.30/2.1.2-provenance.txt @@ -0,0 +1,24 @@ +2.1.2 Integration of provenance tracking mechanism +Status: 66% +Synopsis: + +In this quarter, we have continued our collaborative work with Yong Chen and Dong Dai of Texas Tech University on a prototype provenance tracking mechanism based on the property graph abstraction explored in our previous Parallel Data Storage Workshop paper [1]. In the previous report, we had identified three primary challenges in using such an abstraction: + +1) metadata ingestion from HPC applications +2) efficient traversal of large-scale metadata graphs +3) metadata graph partitioning for graphs with properties identified in [1] + +We have made the following progress on these challenges, informing our prototype efforts: + +1) We are currently considering both batch metadata operations along the lines of the recent IndexFS/BatchFS works [?,?] and finer-grained operations. Different metadata workloads require different levels of consistency and time until the operation is visible to the system as a whole. For instance, "create" events in a large-scale application using a file-per-process I/O strategy need not require immediate data visibility and consistency, as it can be reasonably assumed that, for the lifetime of the application, the corresponding clients are the only accessors of the files. Checkpointing workloads could also fit under this category, though with higher requirements on resulting durability in the face of application failures. On the other hand, applications monitoring file changes in real time would need high consistency and visibility requirements (TODO: example?). Relaxing these requirements can allow us to design "opt-in" optimizations that sacrifice immediate consistency/v isibility for performance, such as merging large numbers of metadata operations into a single bulk update into storage. +2) We have developed an initial design for server-side metadata graph traversal. As observed in [1], our usage of rich metadata graphs can be characterized by imbalanced vertex/edge distributions, long traversal lengths, and concurrent workloads, each of which has the potential to introduce or exacerbate the problem of malfunctioning/overloaded servers slowing down overall metadata query times. In response, we have developed an asynchronous traversal methodology, by which one server serves as a per-query coordinator for the others. In addition, we have developed a simple traversal query language based on successive edge and vertex filters in order to succinctly represent traversal queries of interest. An example query is shown in the following figure, along with vertex/edge data transfers (shown as black circles) and coordinator state transfers (shown as green circles). The traversal can be described as "starting from user 'userA', find, across all jobs (vertices reachable b y the 'run' edge) in a given time range, that read files with the extension '.txt'." Evaluation of the methods are ongoing, and we plan to submit a paper on our findings in the near future. + +(TODO: put in figure asynctraversal-2_cropped.jpeg) + +3) (TODO: discuss with Dong the tradeoffs and initial plan. I've reached out to Dong for some rough notes and figures from meetings that we can put here) + +In the next quarter, we plan to finish the related studies and turn focus to integrating with Triton, either as a standalone service coordinating with Triton (metadata server as-a-client) or as a protocol within Triton. + +[1] TODO: metadata graph pub +[2] TODO: IndexFS +[3] TODO: BatchFS diff --git a/reports/triton-quarterly/2015.03.30/asynctravel-2_cropped.jpeg b/reports/triton-quarterly/2015.03.30/asynctravel-2_cropped.jpeg new file mode 100644 index 0000000..935e32e Binary files /dev/null and b/reports/triton-quarterly/2015.03.30/asynctravel-2_cropped.jpeg differ hooks/post-receive --

1 0