March 2012 - Triton-commits - mailman3.cels.anl.gov

Triton-private Repository branch, master, updated. e85b2f6d75c8333e5b1d39d6818149bb4c27471f
by noreply＠mcs.anl.gov 14 Mar '12

14 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via e85b2f6d75c8333e5b1d39d6818149bb4c27471f (commit) from 42c843a30197be314dbc2c9a36f98d453e1aab47 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e85b2f6d75c8333e5b1d39d6818149bb4c27471f Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Wed Mar 14 22:23:41 2012 -0500 aesop paper: tweak figure 3 caption ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/evaluation.tex | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/evaluation.tex b/papers/2011/aesop/evaluation.tex index 23f01c4..f799673 100644 --- a/papers/2011/aesop/evaluation.tex +++ b/papers/2011/aesop/evaluation.tex @@ -225,8 +225,8 @@ fundamental programming language problem. \label{fig:writenull} } \caption{Runtime performance for each test case.\\ -Client instances are scaled from 16 to 1024.\\ -(Each graph has a distinct Y scale)} +\footnotesize x-axis is the number of client instances.\\ +\footnotesize (Each graph has a distinct y-axis)} \end{figure*} The test client also measures the latency of each individual request and then hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. 42c843a30197be314dbc2c9a36f98d453e1aab47
by noreply＠mcs.anl.gov 14 Mar '12

14 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 42c843a30197be314dbc2c9a36f98d453e1aab47 (commit) from 1226a4513eb3d1af0a2df0a688817cd92ba0682f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 42c843a30197be314dbc2c9a36f98d453e1aab47 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Wed Mar 14 22:17:54 2012 -0500 aesop paper: change Fig 3 plots so they have errors bars and conserve more space ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/fig/Makefile | 8 +++++++- papers/2011/aesop/fig/gendata.sh | 14 +++++++++----- papers/2011/aesop/fig/read-null-ops.gnuplot | 6 ++++-- papers/2011/aesop/fig/read-ops.gnuplot | 7 +++++-- papers/2011/aesop/fig/write-null-ops.gnuplot | 6 ++++-- papers/2011/aesop/fig/write-ops.gnuplot | 7 +++++-- 6 files changed, 34 insertions(+), 14 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/fig/Makefile b/papers/2011/aesop/fig/Makefile index 7617a61..66d9db0 100644 --- a/papers/2011/aesop/fig/Makefile +++ b/papers/2011/aesop/fig/Makefile @@ -30,8 +30,14 @@ clean:: @echo 'set terminal "png" transparent large font "/usr/share/fonts/truetype/ttf-liberation/LiberationSans-Regular.ttf, 12"; set output "$@"' | gnuplot - $< +read-ops.pdf: read-ops.gnuplot read-ops.data + @echo 'set terminal "pdf" size 4in,2.1in; set output "$@"' | gnuplot - $< + +write-ops.pdf: write-ops.gnuplot write-ops.data + @echo 'set terminal "pdf" size 4in,2.1in; set output "$@"' | gnuplot - $< + %.pdf: %.gnuplot %.data - @echo 'set terminal "pdf"; set output "$@"' | gnuplot - $< + @echo 'set terminal "pdf" size 5in,3in; set output "$@"' | gnuplot - $< %.data: @./gendata.sh $* diff --git a/papers/2011/aesop/fig/gendata.sh b/papers/2011/aesop/fig/gendata.sh index a4f24cd..de66144 100755 --- a/papers/2011/aesop/fig/gendata.sh +++ b/papers/2011/aesop/fig/gendata.sh @@ -40,7 +40,7 @@ then # Max Time Data # create column oriented data - echo "#${base} aesop thread thread-nb thread-per-op thread-pool event" > ${test}.data; + echo "#clients aesop-median aesop-min aesop-max thread-median thread-min thread-max thread-nb-median thread-nb-min thread-nb-max thread-per-op-median thread-per-op-min thread-per-op-max thread-pool-median thread-pool-min thread-pool-max event-median event-min event-max" > ${test}.data; for nodes in 1 8 16 32 64; do @@ -49,8 +49,10 @@ then for type in aesop thread thread-nb thread-per-op thread-pool event; do - val=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | grep "max:" | perl -ne 'if (/max:(\S+)/) { print "$1\n"; }' | sort -n | head -3 | tail -1) - if [ -z ${val} ]; then + medval=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | grep "max:" | perl -ne 'if (/max:(\S+)/) { print "$1\n"; }' | sort -n | head -3 | tail -1) + minval=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | grep "max:" | perl -ne 'if (/max:(\S+)/) { print "$1\n"; }' | sort -n | tail -1 ) + maxval=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | grep "max:" | perl -ne 'if (/max:(\S+)/) { print "$1\n"; }' | sort -n | head -1) + if [ -z ${medval} ]; then val="-"; else if [ ${base} = "read" -o ${base} = "write" ]; @@ -62,9 +64,11 @@ then else op=0 fi - val=$(echo "(${nodes}*16*${op}) / ${val}" | bc) + medval=$(echo "scale=1;(${nodes}*16*${op}) / ${medval}" | bc) + minval=$(echo "scale=1;(${nodes}*16*${op}) / ${minval}" | bc) + maxval=$(echo "scale=1;(${nodes}*16*${op}) / ${maxval}" | bc) fi - echo -n "${val} " >> ${test}.data; + echo -n "${medval} ${minval} ${maxval} " >> ${test}.data; done echo "" >> ${test}.data diff --git a/papers/2011/aesop/fig/read-null-ops.gnuplot b/papers/2011/aesop/fig/read-null-ops.gnuplot index ab5e16c..7b53015 100644 --- a/papers/2011/aesop/fig/read-null-ops.gnuplot +++ b/papers/2011/aesop/fig/read-null-ops.gnuplot @@ -56,7 +56,8 @@ unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 +set style histogram errorbars gap 2 lw 2 title offset character 0, 0, 0 unset logscale #set logscale y 10 set offsets 0, 0, 0, 0 @@ -150,5 +151,6 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-null-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +#plot 'read-null-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'read-null-ops.data' using 2:3:4:xtic(1) title "aesop", '' u 5:6:7 ti "thread-per-client", '' u 8:9:10 ti "thread-per-client-nb", '' u 11:12:13 ti "thread-per-op", '' u 14:15:16 ti "thread-pool", '' u 17:18:19 ti "event" # EOF diff --git a/papers/2011/aesop/fig/read-ops.gnuplot b/papers/2011/aesop/fig/read-ops.gnuplot index 58bdda3..0b4864c 100644 --- a/papers/2011/aesop/fig/read-ops.gnuplot +++ b/papers/2011/aesop/fig/read-ops.gnuplot @@ -51,12 +51,14 @@ set key title "" set key outside bottom center horizontal Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 +set style histogram errorbars gap 2 lw 2 title offset character 0, 0, 0 unset logscale #set logscale y 10 set offsets 0, 0, 0, 0 @@ -150,5 +152,6 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +#plot 'read-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'read-ops.data' using 2:3:4:xtic(1) title "aesop", '' u 5:6:7 ti "thread-per-client", '' u 8:9:10 ti "thread-per-client-nb", '' u 11:12:13 ti "thread-per-op", '' u 14:15:16 ti "thread-pool", '' u 17:18:19 ti "event" # EOF diff --git a/papers/2011/aesop/fig/write-null-ops.gnuplot b/papers/2011/aesop/fig/write-null-ops.gnuplot index 0194584..c7b487d 100644 --- a/papers/2011/aesop/fig/write-null-ops.gnuplot +++ b/papers/2011/aesop/fig/write-null-ops.gnuplot @@ -56,7 +56,8 @@ unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 +set style histogram errorbars gap 2 lw 2 title offset character 0, 0, 0 unset logscale #set logscale y 10 set offsets 0, 0, 0, 0 @@ -150,5 +151,6 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'write-null-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +#plot 'write-null-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'write-null-ops.data' using 2:3:4:xtic(1) title "aesop", '' u 5:6:7 ti "thread-per-client", '' u 8:9:10 ti "thread-per-client-nb", '' u 11:12:13 ti "thread-per-op", '' u 14:15:16 ti "thread-pool", '' u 17:18:19 ti "event" # EOF diff --git a/papers/2011/aesop/fig/write-ops.gnuplot b/papers/2011/aesop/fig/write-ops.gnuplot index ad00bed..900f978 100644 --- a/papers/2011/aesop/fig/write-ops.gnuplot +++ b/papers/2011/aesop/fig/write-ops.gnuplot @@ -51,12 +51,14 @@ set key title "" set key outside bottom center horizontal Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 +set style histogram errorbars gap 2 lw 2 title offset character 0, 0, 0 unset logscale #set logscale y 10 set offsets 0, 0, 0, 0 @@ -150,5 +152,6 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'write-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +#plot 'write-ops.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'write-ops.data' using 2:3:4:xtic(1) title "aesop", '' u 5:6:7 ti "thread-per-client", '' u 8:9:10 ti "thread-per-client-nb", '' u 11:12:13 ti "thread-per-op", '' u 14:15:16 ti "thread-pool", '' u 17:18:19 ti "event" # EOF hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. 1226a4513eb3d1af0a2df0a688817cd92ba0682f
by noreply＠mcs.anl.gov 13 Mar '12

13 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 1226a4513eb3d1af0a2df0a688817cd92ba0682f (commit) from 2d6f626ce59ab4ebe0bcdcb2e92b664f9a1f9d44 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1226a4513eb3d1af0a2df0a688817cd92ba0682f Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Mar 13 23:43:43 2012 -0500 aesop paper: update Figure 3 caption to add notes on x-axis and y-scales ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/aesop.tex | 1 + papers/2011/aesop/evaluation.tex | 4 +++- 2 files changed, 4 insertions(+), 1 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/aesop.tex b/papers/2011/aesop/aesop.tex index 9ce54b6..865318c 100644 --- a/papers/2011/aesop/aesop.tex +++ b/papers/2011/aesop/aesop.tex @@ -1,3 +1,4 @@ +\PassOptionsToPackage{justification=centering}{caption} \documentclass[10pt, conference, compsocconf]{IEEEtran} \usepackage{hyperref} diff --git a/papers/2011/aesop/evaluation.tex b/papers/2011/aesop/evaluation.tex index 474433b..23f01c4 100644 --- a/papers/2011/aesop/evaluation.tex +++ b/papers/2011/aesop/evaluation.tex @@ -224,7 +224,9 @@ fundamental programming language problem. \includegraphics[keepaspectratio,width=0.45\textwidth]{fig/write-null-ops.pdf} \label{fig:writenull} } - \caption{Runtime performance for four test cases} + \caption{Runtime performance for each test case.\\ +Client instances are scaled from 16 to 1024.\\ +(Each graph has a distinct Y scale)} \end{figure*} The test client also measures the latency of each individual request and then hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. 2d6f626ce59ab4ebe0bcdcb2e92b664f9a1f9d44
by noreply＠mcs.anl.gov 13 Mar '12

13 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 2d6f626ce59ab4ebe0bcdcb2e92b664f9a1f9d44 (commit) from 3ba288ec312b1621cf063753bdad8b844c2bad52 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 2d6f626ce59ab4ebe0bcdcb2e92b664f9a1f9d44 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Mon Mar 12 00:52:31 2012 -0500 Full pass over the paper - includes Kevin's recent fixes ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/aesop.tex | 60 +++-- papers/2011/aesop/conclusion.tex | 54 ++-- papers/2011/aesop/context.tex | 345 +++++++++++------------- papers/2011/aesop/evaluation.tex | 505 +++++++++++++++++----------------- papers/2011/aesop/implementation.tex | 86 ++++++ papers/2011/aesop/intro.tex | 183 ++++++------- papers/2011/aesop/model.tex | 212 ++++++++------- papers/2011/aesop/related.tex | 145 ++++++---- 8 files changed, 852 insertions(+), 738 deletions(-) create mode 100644 papers/2011/aesop/implementation.tex Diff of changes: diff --git a/papers/2011/aesop/aesop.tex b/papers/2011/aesop/aesop.tex index d8fc6e4..9ce54b6 100644 --- a/papers/2011/aesop/aesop.tex +++ b/papers/2011/aesop/aesop.tex @@ -1,11 +1,11 @@ - \documentclass[10pt, conference, compsocconf]{IEEEtran} +\usepackage{hyperref} \usepackage[pdftex]{graphicx} \usepackage{color} +\usepackage{multirow} % \usepackage{sidecap} \usepackage{listing} -\usepackage{hyperref} \usepackage{listings} \usepackage{subfig} \usepackage{xspace} @@ -20,10 +20,19 @@ frame=single, language=C, captionpos=b, columns=fullflexible, -basicstyle=\small +morekeywords={__blocking,pwait,pbranch,pbreak}, +numbers=left, +basicstyle=\footnotesize\ttfamily, +breaklines=true, +framexleftmargin=0em, +boxpos=c, +resetmargins=true, +xleftmargin=6ex %basicstyle=\footnotesize } +\renewcommand*\thelstnumber{{\oldstylenums{\the\value{lstnumber}}}} + %\usepackage{algorithm} %\usepackage[noend]{algorithmic} @@ -39,14 +48,15 @@ basicstyle=\small % correct bad hyphenation here \hyphenation{op-tical net-works semi-conduc-tor} -\DeclareRobustCommand{\code}[1]{\nolinkurl{#1}} +\DeclareRobustCommand{\code}[1] + {\lstinline[breaklines=true,basicstyle=\ttfamily]{#1}} %\newcommand{\todo}[1]{ {\textcolor{red} { TODO: #1 }}} -\newcommand{\woz} [1]{ {\textcolor{blue} { Wozniak: #1 }}} +\newcommand{\woz} [1] {\todo{ Wozniak: #1 }} % fix issue with class and todo notes - does not affect paper margins -\setlength{\marginparwidth}{1.4cm} +%\setlength{\marginparwidth}{1.4cm} \makeatletter \newif\ifsc@active @@ -99,35 +109,35 @@ High Performance Computing (HPC) and distributed systems rely on a diverse collection of system software to provide application services, including file systems, schedulers, web services, and fault detectors. Such system software services must manage highly concurrent requests, interact with a wide range of -resources, and scale well in order to be successful. -Unfortunately, there is no single programming model for distributed system -software that offers optimal performance in all cases. In addition, many +resources, and scale well in order to be successful. Unfortunately, there is +no single programming model for distributed system software that offers +optimal performance and productivity for all these tasks. In addition, many concurrent server architectures introduce software complexity that interferes with algorithm development and hinders software maintenance and portability. -While numerous languages and language extensions have been developed in recent years -to simplify parallel computation, they do not address the challenges of -distributed system software in which concurrency control involves a variety of -hardware and network devices, not just computational resources. +While numerous libraries, languages and language extensions have been +developed in recent years to simplify parallel computation, they do not +address the challenges of distributed system software in which concurrency +control involves a variety of hardware and network devices, not just +computational resources. In this work we present \aesop, a new programming language and programming model designed to implement distributed system software with high development productivity and run time efficiency. \aesop is a superset of the C language -that describes blocks of code to be executed concurrently without dictating -the threading or event model that will be used to provide that -concurrency.\todo{leave out threading and event?} This decoupling of -algorithm description and run time implementation enables system software to -adjust to different architectures, device APIs, and workloads without any -change to the core algorithm implementation. \aesop also provides a set of -language constructs that provide more expressive control over concurrent -execution than is typically found in multithreaded or event-driven programming -models. +that describes blocks of code to be executed concurrently without dictating if +that concurrency will be provided by a threading, event or other model. This +decoupling of algorithm description and run time implementation enables system +software to adjust to different architectures, device APIs, and workloads +without any change to the core algorithm implementation. \aesop also provides +a set of language constructs that provide more expressive control over +concurrent execution than is typically found in multithreaded or event-driven +programming models. We evaluate \aesop by implementing a simple file server and comparing its performance, memory efficiency, and developer productivity to several thread-based and event-based implementations. \aesop is shown to provide competitive (and in some cases superior) performance to traditional distributed system software development models while at the same time reducing -code complexity and providing constructs that enhance developer productivity. +code complexity and enhancing developer productivity. \end{abstract} \section{Introduction} @@ -142,9 +152,9 @@ code complexity and providing constructs that enhance developer productivity. \input{model} -%\section{The Aesop Language} +\section{Implementation} -%\input{language} +\input{implementation} \section{Evaluation} diff --git a/papers/2011/aesop/conclusion.tex b/papers/2011/aesop/conclusion.tex index f56b26b..2f7c015 100644 --- a/papers/2011/aesop/conclusion.tex +++ b/papers/2011/aesop/conclusion.tex @@ -1,40 +1,42 @@ \label{section:conclusion} In this work we have introduced a new programming model and programming -language, known as \aesop, that was designed with the explicit goal of -supporting distributed system software development. Its primary feature is +language, known as \aesop. \aesop was designed with the explicit goal of +supporting distributed system software development. Its primary advantage is that it allows system software developers to express concurrency without -dictating the threading or event model used to provide that concurrency. It -also provides a number of language constructs that simplify common system -software development patterns. Numerous languages and language -extensions have been developed in recent years to simplify parallel -computation, but \aesop is unique in that it addresses the challenges of -distributed system software in which concurrency control involves a variety of -hardware and network devices, not just computational resources. - -\color{red}TODO: sprinkle in some of the important numbers from the -evaluation section here. \color{black} - -From our experimental evaluation, we see that Aesop offers competitive, -and in some cases superior, runtime performance to the most common system -software concurrency models in use today. We expect Aesop to continue -to perform well at even larger scales than those evaluated in this study -due to the fact that it does not rely on mapping concurrent execution -paths to dedicated operating system threads. We have also found that -the Aesop language reduces code complexity while also offering advanced -primitives such as cancellation of execution paths and control -of variable scope that are not available in any other system software +dictating the model (for example threads or event handling) used to provide +that concurrency. It also provides a number of language constructs +simplifying common system software development patterns. Numerous languages +and language extensions have been developed in recent years to simplify +parallel computation, but \aesop is unique in that it addresses the challenges +of distributed system software in which concurrency control goes beyond +computational resources but also includes a variety of hardware and +network devices. + +Using a simple network server for evaluation, we demonstrate that \aesop +offers competitive, and in some cases superior, runtime performance to the +most common system software concurrency models in use today. When considering +programmer productivity, \aesop performs equally well, mainly due to the fact +that is has been specifically designed to implement highly scalable +distributed services. + +We expect \aesop to continue to perform well at even larger scales than those +evaluated in this study due to the fact that it does not rely on mapping +concurrent execution paths to dedicated operating system threads. We have +also found that the \aesop language reduces code complexity while also +offering advanced primitives such as cancellation of execution paths and +control of variable scope that are not available in any other system software development environment. -We will continue to evaluate and evolve the Aesop language as we use it +We will continue to evaluate and evolve the \aesop language as we use it to implement future production HPC storage services. As part of this work we will likely extend the language to include additional features. One example, which exists in prototype form already, is remote procedure -call (RPC) framework that uses Aesop to manage concurrent communication, +call (RPC) framework that uses \aesop to manage concurrent communication, timeout, and retry functionality in a transparent manner. We also plan to improve debugging functionality by providing mechanisms to trace control -flow through Aesop blocking functions. This will allow us to generate -more meaningful stack traces that map directly to the original Aesop +flow through \aesop blocking functions. This will allow us to generate +more meaningful stack traces that map directly to the original \aesop code rather than the underlying C code that it has been translated to. %- Need to fix some parser issues diff --git a/papers/2011/aesop/context.tex b/papers/2011/aesop/context.tex index ca8e350..ff047e1 100644 --- a/papers/2011/aesop/context.tex +++ b/papers/2011/aesop/context.tex @@ -43,64 +43,70 @@ % \subsection{Distributed system software} -\begin{figure} - %\vskip -0.25in - \centering - \includegraphics[keepaspectratio,width=0.45\textwidth]{state-machines.pdf} - \caption{Concurrency in file system server. - \label{fig:sm} - } -\end{figure} - -Large-scale distributed and parallel systems rely on distributed -system software to provide scalable access to persistent services. -For example, a parallel file system aggregates distributed storage -resources provided by multiple servers into a unified name space. -Each individual service program expects many concurrent accesses from -multiple clients, and in turn launches multiple overlapping operations -such as network transmissions or disk drive operations. A large -system consisting of $O(100)$ servers could service $O(1000)$ clients, -each performing multiple simultaneous requests. Each request typically -requires the server to perform $O(10)$ asynchronous disk or network -operations. {\color{red}KH: how do we backup that each request typically takes -O(10) operations?} - -%% \todo{This might be a good place for a mini plot or table.} - -\begin{table}[h] +\begin{table}[tblr] \caption{Representative scale of current file systems} - \label{tab:counts} \begin{tabular}{|l|l|r|r|} \hline - \textbf{Installation} & \textbf{File system} & - \textbf{Servers} & \textbf{CPUs$^\ast$} \\ + \multirow{2}{*}{\textbf{Installation}} + & \multirow{2}{*}{\textbf{File System}} + & \multicolumn{2}{c|}{\textbf{Number of Nodes}} \\ + \cline{3-4} + & & \textbf{File System} & \textbf{Client} \\ \hline - ALCF Intrepid & GPFS, PVFS & $128$ & $160,000$ \\ - NCCS Spider & Lustre & $672$ & $250,000$ \\ + ALCF Intrepid & GPFS, PVFS & $128$ & $\sim 40,000$ \\ + NCCS Spider & Lustre & $672$ & $\sim 40,000$ \\ Aggressive Strawman~\cite{Kogge_2008} & - ? & $\sim 2 \times 10^5$ & $\sim 10^8$ \\ + ? & $\sim 2 \times 10^5$ & $\sim 223,000$ \\ \hline \end{tabular} \\ \ \\ % \vspace{4mm} - $^\ast$In practice, I/O is typically aggregated through a limited - number \\ of clients + %$^\ast$In practice, I/O is typically aggregated through a limited + %number \\ of clients + \label{tab:scale} \end{table} %% I spent a lot of time looking for the OST count on the K-computer %% but failed. -Justin + +Large-scale distributed and parallel systems rely on distributed system +software to provide scalable access to persistent services. For example, a +parallel file system aggregates distributed storage resources provided by +multiple servers into a unified name space. Each individual service program +expects many concurrent accesses from multiple clients, and in turn launches +multiple overlapping operations such as network transmissions or disk drive +operations. Table~\ref{tab:scale} details the configuration of some existing +and predicted deployments of parallel file systems. A large parallel file +system consisting of $O(100)$ nodes currently typically services $O(10000)$ +client nodes, with each client submitting multiple concurrent requests. Each +request typically requires the server to perform $O(10)$ disk or network +operations. + +%% \todo{This might be a good place for a mini plot or table.} + \subsection{Motivation: Concurrency in file system services} -Consider the file server depicted in Figure~\ref{fig:sm}. The -operation starts when the client issues a call to the server, -triggering the top receive. The server then simultaneously issues -four concurrent operations: 1) a send to peer server to -forward data to a replica, \todo{removed non-blocking here to avoid term -confusion} 2) a receive to accept -the response from the replica server, 3) a timer call to -detect a timeout if necessary, and 4) a -sequence of metadata operations. Finally, when all four operations -complete, the result is posted to the client. +Consider the file server depicted in Figure~\ref{fig:sm}. The operation +starts when the client issues a request to the server, triggering the top +receive. The server then simultaneously issues four concurrent operations: a +send to peer server to forward data to a replica, a receive to accept the +response from the replica server, a timer call to ensure the request completes +within a reasonable time, and a sequence of metadata operations. Finally, +when all operations complete before the timer expires, the timer is cancelled +and the result is posted to the client. However, if the timeout is reached +before the other tasks complete, all unfinished operations are cancelled and +an error is returned to the client. + + +\begin{figure} + %\vskip -0.25in + \centering + \includegraphics[keepaspectratio,width=0.45\textwidth]{state-machines.pdf} + \caption{Concurrency in file system server. + \label{fig:sm} + } +\end{figure} + This file system server algorithm example illustrates that concurrency arises not only from simultaneous client access, but also from the @@ -116,38 +122,41 @@ complicates its implementation. %% large number of concurrent requests, each request results in N %% device-level operations. -\subsection{Programming challenges} - -\todo{Suddenly imperative programming is brought in. What does this subsection -add (over the introduction)?} -Imperative programming is the {\it de facto} standard for system -software of this nature for historical and performance reasons. All -cluster and parallel file systems in wide use today are written in C -or C++~\cite{GPFS_2002,Lustre_2002,Panasas_2008,Ceph_2006}. -Imperative programming, by definition, places a strict ordering on the -steps that must be performed by a program. Multithreading loosens the -ordering constraints of imperative programming, allowing multiple -programs to proceed concurrently within a shared memory space. But -threads disrupt the logical ordering of an imperative program, and -often add complexity to the program with synchronization points that -are only necessary because of the non-determinism introduced by -threads. + +% not sure what the text below adds; commented out for now + +%\subsection{Programming challenges} +% +%\todo{Suddenly imperative programming is brought in. What does this subsection +%add (over the introduction)?} +%Imperative programming is the de facto standard for system +%software of this nature for historical and performance reasons. All +%cluster and parallel file systems in wide use today are written in C +%or C++~\cite{GPFS_2002,Lustre_2002,Panasas_2008,Ceph_2006}. +%Imperative programming, by definition, places a strict ordering on the +%steps that must be performed by a program. Multithreading loosens the +%ordering constraints of imperative programming, allowing multiple +%programs to proceed concurrently within a shared memory space. But +%threads disrupt the logical ordering of an imperative program, and +%often add complexity to the program with synchronization points that +%are only necessary because of the non-determinism introduced by +%threads. %% \emph{...this is especially true for preemptive multithreading, where %% a context switching occurs in a non-deterministic fashion...} %% todo: an overview of what a "system software algorithm" is (ie, stuff like %% aggregation, caching, pipelining, fault detection etc) to given a sense of %% what we intend to build with aesop beyond simple remote storage requests -%% See below: +%%% See below: -Expressing concurrency in systems software with traditional tools is -challenging due to the highly asynchronous nature of \emph{systems - software algorithms}, which perform common tasks such as aggregation -of operations, caching, and pipelining. These operations require -complex concurrency and synchronization mechanisms. Another -challenging use case is fault detection, in which an operation posted -to an underlying software layer exceeds a time limit, resulting in a branch to a -fault response code path. +%Expressing concurrency in systems software with traditional tools is +%challenging due to the highly asynchronous nature of \emph{systems +% software algorithms}, which perform common tasks such as aggregation +%of operations, caching, and pipelining. These operations require +%complex concurrency and synchronization mechanisms. Another +%challenging use case is fault detection, in which an operation posted +%to an underlying exceeds a time limit, resulting in a branch to a +%fault response code path. %% In the server %% market, computers support many independent requests from a variety @@ -166,13 +175,13 @@ fault response code path. %can handle thousands of requests concurrently, File system servers %support ..., and peer-to-peer servers support .... -While our work targets high-performance systems, commodity systems are -faced with similar challenges. Example systems include web and other -static content servers, which are read-intensive and thus highly -cacheable.\todo{caching: not relevant here} NFS~\cite{NFS_1985} and other typical distributed file -systems are a more challenging problem from a consistency perspective -as common applications rely on standardized filesystem consistency -semantics for correctness.\todo{consistency: relevance?} +%While our work targets high-performance systems, commodity systems are +%faced with similar challenges. Example systems include web and other +%static content servers, which are read-intensive and thus highly +%cacheable.\todo{caching: not relevant here} NFS~\cite{NFS_1985} and other typical distributed file +%systems are a more challenging problem from a consistency perspective +%as common applications rely on standardized filesystem consistency +%semantics for correctness.\todo{consistency: relevance?} % Standard Practice: % Use threads, locks, condition variables @@ -186,43 +195,50 @@ semantics for correctness.\todo{consistency: relevance?} % Discuss areas where its common: %high-concurrency servers: web, file system, database systems -\subsection{Device interfaces} +\subsection{Device Interfaces} + +%Imperative programming is the de facto standard for system +%software of this nature for historical and performance reasons. All +%cluster and parallel file systems in wide use today are written in C +%or C++~\cite{GPFS_2002,Lustre_2002,Panasas_2008,Ceph_2006}. +% %% todo: an overview of what variety devices we may need to interface %% with, how diverse their APIs are, and how many of them there might %% be (ie, enterprise storage, commodity storage, TCP/IP sockets, HPC %% networks, MPI, etc.) -Any language feature designed to ease managing asynchronous -devices must take into account the broad array of relevant underlying -devices and their interface models. -\todo{are we trying to say that there are many different API -models? Why 'usually in C'? Why contrast non-blocking with event?} -To enable I/O concurrency, -high-level, high performance storage and networking APIs typically -provide non-blocking interfaces, but lower-level APIs provide -event-driven interfaces, all usually in C. - -Non-blocking interfaces follow an \emph{event-driven model}, -\todo{see text above; now non-blocking = event?} where a -request for an \emph{I/O operation} is made to the device through a -function call, which returns without waiting for completion (the -function does not block). The program receives completion of the I/O -operation (the event) through other notification paths, such as -through a polling function or asynchronous function callbacks. -Non-blocking interfaces achieve concurrency by enabling the posting of multiple I/O operations -without waiting for their completion. - While a non-blocking interface -allows the device to receive and optimize many requests, it hinders -expressiveness of the concurrent program. Event-driven software is -inherently difficult to write, understand and debug, primarily because -the logical tasks performed by the program can no longer be expressed -in an imperative manner. Instead, concurrent programs must implement -their own stack management; manually keeping track of the state of a -task as it transitions from one I/O operation to the next. This -process, often refered to as \emph{stack ripping}~\cite{krohn2007ecm}, -has encumbered many programmers responsible with writing highly -concurrent server software. +A programming model designed to ease managing asynchronous device operations +must take into account the broad array of relevant underlying devices and +their interface models. For a device exporting a blocking interface, function +calls do not return until the request described by the call is completed. +Concurrency is typically achieved by calling multiple functions simultaneously +from multiple threads. However, for scheduling and efficiency reasons +(described in section~\ref{sec:intro}), high performance storage and +networking APIs typically provide non-blocking interfaces to achieve I/O +concurrency. + +Non-blocking interfaces follow an \emph{event-driven model}, where a request +for an \emph{I/O operation} is made to the device through a function call, +which returns without waiting for completion (the function does not block). +The program receives completion of the I/O operation (the event) through other +notification paths, such as through a polling function or asynchronous +function callbacks. Non-blocking interfaces achieve concurrency by enabling +the posting of multiple I/O operations without waiting for their completion. + + + +%While a non-blocking interface +%allows the device to receive and optimize many requests, it hinders +%expressiveness of the concurrent program. Event-driven software is +%inherently difficult to write, understand and debug, primarily because +%the logical tasks performed by the program can no longer be expressed +%in an imperative manner. Instead, concurrent programs must implement +%their own stack management; manually keeping track of the state of a +%task as it transitions from one I/O operation to the next. This +%process, often refered to as \emph{stack ripping}~\cite{krohn2007ecm}, +%has encumbered many programmers responsible with writing highly +%concurrent server software. % Async APIs % Low-level storage and networking APIs have asynchronous, @@ -231,31 +247,29 @@ concurrent server software. %% Infiband verbs API, IT-API (Interconnect Transport API), %% Myricom MX, -\subsection*{Summary of non-blocking APIs} - -File system services make heavy use of non-blocking APIs for both -network and disk access. - -Network sockets offers multiple potential APIs, including non-blocking -APIs and polling. Use of InfiniBand~\cite{IBVerbs,balaji2006azc} -through the {\tt libibverbs} API offers a post/poll interface to -send/receive queues~\cite{ITAPI,RDMA_2010}. -Myrinet~\cite{Myrinet_1995} provides a rich API through the MX -interface~\cite{MX_Guide_2006} that offers asynchronous send/receive -operations with test, probe, and callback registration. However, -straightforward use of the callbacks block the network progress -thread. The IBM Deep Computing Messaging Framework -(DCMF)~\cite{BGP_RMA_2008} primarily offers non-blocking calls for -send/receive and one-sided put/get communication. -MPI-2~\cite{gropp1999ump,snir1995mcr} offers non-blocking messaging -calls for point-to-point communication. - -Unix systems have traditionally offered the select/poll paradigm, in -which file descriptors are collected into a data structure; the -waiting process then makes a library call that does not return until -some file descriptor makes progress (or an exception occurs). -epoll~\cite{EPOLL_WWW}, available in Linux, offers a similar model but -is typically more scalable for large numbers of file descriptors. +%\subsection*{Summary of non-blocking APIs} + +%File system services make heavy use of non-blocking APIs for both +%network and disk access. + +Network sockets offers multiple potential APIs, including non-blocking APIs +and polling. Use of InfiniBand~\cite{IBVerbs,balaji2006azc} through the {\tt +libibverbs} API offers a post/poll interface to send/receive +queues~\cite{ITAPI,RDMA_2010}. Myrinet~\cite{Myrinet_1995} provides a rich +API through the MX interface~\cite{MX_Guide_2006} that offers asynchronous +send/receive operations with test, probe, and callback registration. However, +straightforward use of the callbacks block the network progress thread. The +IBM Deep Computing Messaging Framework (DCMF)~\cite{BGP_RMA_2008} primarily +offers non-blocking calls for send/receive and one-sided put/get +communication. MPI-2~\cite{gropp1999ump,snir1995mcr} offers non-blocking +messaging calls for point-to-point communication. + +Unix systems have traditionally offered the select/poll paradigm, in which +file descriptors are collected into a data structure; the waiting process then +makes a function call that does not return until some file descriptor makes +progress (or an exception occurs). Epoll~\cite{EPOLL_WWW}, available in +Linux, offers a similar model but is typically more scalable for large numbers +of file descriptors. %% \color{red}PHC: the following paragraph needs some work. You can't %% actually use select, poll, or epoll on files in practice. Maybe @@ -266,20 +280,18 @@ is typically more scalable for large numbers of file descriptors. %% Addressed. -Justin -Disk operations also have a range of non-blocking access methods. -POSIX AIO~\cite{POSIX_AIO_2006} offers multiple models for controlling -asynchronous I/O, including the ability to supply a signal or callback -function for completion notification. Additionally, the user may -manually place disk operations in separate threads to enhance -concurrency. +Disk operations also have a range of non-blocking access methods. POSIX +AIO~\cite{POSIX_AIO_2006} offers multiple models for controlling asynchronous +I/O, including the ability to supply a signal or callback function for +completion notification. Additionally, the user may manually place disk +operations in separate threads to enhance concurrency. -Databases are commonly used for file system metadata and the efficient -use of them is critical. Berkeley~DB~\cite{BDB_1999} does not provide -a non-blocking API, but wrapper libraries have been used to achieve -this. Postgres offers an asynchronous interface~\cite{postgresAsync} -in its C API that allows database operations to be posted -asynchronously; a separate library function tests if a complete -operation may be accessed. +Databases are commonly used for file system metadata and the efficient use of +them is critical. Berkeley~DB~\cite{BDB_1999} does not provide a non-blocking +API, but wrapper libraries have been used to achieve this. Postgres offers an +asynchronous interface~\cite{postgresAsync} in its C API that allows database +operations to be posted asynchronously; a separate library function tests if a +complete operation may be accessed. % Discuss continued use of C/C++ in systems software: % Operating systems: Linux, BSD, Windows @@ -301,48 +313,3 @@ operation may be accessed. %TODO: an overview of the scale we anticipate (ie, cite some projections about %how many nodes and disks etc. we might see at exascale) -\subsection{PVFS state machines} - -The \aesop language described in this work builds upon lessons learned -from the event-driven state machine model used in the PVFS file -system~\cite{pvfs-web}. -In PVFS State Machines (SM), every client and server operation is -expressed as a state machine using a language that is based on C with -extensions to describe service states and the transitions that link -them. A state is simply a C function. Each state function can -optionally end by submitting an asynchronous network, disk, timer, or -pipeline operation through an API that provides unified notification -of completion events for each device. When one of these operations -completes, a state machine engine maps the operation back to the -appropriate state machine and executes the next state function -depending upon the outcome of the asynchronous operation. Multiple -state machines instances can be active simultaneously, allowing PVFS -to track the state of an arbitrary number of concurrent requests and -make progress on them without the overhead of explicit threading. - -This PVFS state machine model is essentially a formal framework for -event-driven programming. This approach has proved successful -in achieving high performance in large-scale production HPC -environments~\cite{pvfs-bgp-sc09}, but it shares many of the same -developer productivity challenges found in other event-driven models. -First, all algorithms must be split into functions on boundaries according -to the location of asynchronous operations, rather than according to -logical functionality. Secondly, the state machine engine uses a fixed -function prototype for each state machine function, meaning that arguments -are passed between state functions via an opaque struct that is shared -across the entire state machine. This makes automatic depencency checking -between states impossible (e.g., did the previous state function set all -of the struct fields needed by the current state function?) and leads to -a new class of bugs that do not typically appear in traditional C code. -Additionally, PVFS SMs perform all non-asynchronous computation on a -single core, which is an undesirable model given the advent of many-core -servers. Finally, PVFS SM simply does not \emph{look like} traditional -C code, making it difficult to reason about many development challenges. -{\color{red}KH: this statement sounds a bit odd} - -These drawbacks introduce a learning curve for researchers and a -additional maintenance workload for maintainers. In this work, we seek to -learn from the lessons of PVFS SM development and present a programming -model that provides at least as much technical functionality but also allows -file system developers to write algorithms using a more traditional -code organization. diff --git a/papers/2011/aesop/evaluation.tex b/papers/2011/aesop/evaluation.tex index 06503a9..474433b 100644 --- a/papers/2011/aesop/evaluation.tex +++ b/papers/2011/aesop/evaluation.tex @@ -1,187 +1,178 @@ \label{section:evaluation} -In this paper we evaluate \aesop based on three criteria, runtime efficiency, -memory efficiency and progammer productivity. - -\subsection{Experiment} - -To evaluate \aesop we created a comparison of different server -implementations. We designed a simple server that uses TCP messaging. -The server recieves requests from a client, sends or recieves -a payload, and then sends an acknowledgement. -The paylod can be generated or discarded, read from a file or written to a file. - -Each of the example servers used for comparison in this document implement -an identical request protocol and are therefore evaluated using the same -client test harness. The client is a basic C program that uses TCP sockets -to send messages to the server. It uses MPI to coordinate processes and -generate a highly concurrent workload. - -The client will execute in a loop generating a specified number of -operations to the server. The general flow is that each client process sends -a request to the server that contains an optional payload. The client then -waits for the server to send an acknowledgment which also contains an -optional payload. - -Each client records the total amount of time taken to execute its workload -(beginning before the initial connection and ending after receipt of the last -acknowledgment). The time taken by the slowest client is reported as the -aggregate run time. Each client also records the time needed to service each -individual request (from before the request is sent until after the -acknowledgment is received) in order to calculate statistics about -individual request latencies. - -\subsubsection{Experiment - Test Cases} - -There are four distinct test cases run against each server implementation. -All file I/O is done using O\_DIRECT file access mode in order to bypass the -Linux buffer cache and ensure that disk is involved in each I/O transfer. -The number of requests and size of request were chosen to keep the -total runtime of each test case reasonable. +To evaluate \aesop, we implemented a simple TCP network server using a number +of traditional techniques and libraries, and compared the resulting +implementations to the \aesop implementation of the server. We evaluated each +server implementation using three criteria: run time efficiency, memory efficiency +and programmer productivity. + +Each of the servers implements the same request protocol and is evaluated +using the same client test harness. The client is a C program that uses TCP +sockets to send messages to the server. It uses MPI to coordinate processes +and generate a highly concurrent workload. + +%The server recieves requests from a client, sends or recieves +%a payload, and then sends an acknowledgement. +%The paylod can be generated or discarded, read from a file or written to a file. + +%The client will execute in a loop generating a specified number of +%operations to the server. The general flow is that each client process sends +%a request to the server that contains an optional payload. The client then +%waits for the server to send an acknowledgment that also contains an +%optional payload. + +Each client instance records the total amount of time taken to execute its +workload (beginning before the initial connection and ending after receipt of +the last acknowledgment). The time taken by the slowest client is reported as +the aggregate run time. Each client also tracks the time needed to service +each individual request (from the time the request is sent until after the +acknowledgment is received) in order to analyze individual request latency. + +%\subsubsection{Experiment - Test Cases} + +The protocol implemented by our test server recognizes four different +requests. + +%There are four distinct test cases run against each server implementation. +%All file I/O is done using O\_DIRECT file access mode in order to bypass the +%Linux buffer cache and ensure that disk is involved in each I/O transfer. +%The number of requests and size of request were chosen to keep the +%total runtime of each test case reasonable. \paragraph*{Read} -The client sends a request with a file name and a size. The server will -then open the file, read the contents up to the size specified. The server -returns the data with the acknowledgement of the operation. - -The read test had clients each issue 16 requests asking for 4 KiB from -disk. Each client specifies a unique file to be read on each request. All -clients specify unique files. The files are first generated by a script -before the test truns. +The client sends a request containing a file name and a size. The server will +then open the file, read the contents up to the size specified and return the +data to the client. All file I/O is performed using \code{O_DIRECT} mode, in +order to bypass the linux buffer cache and to ensure that the storage device +is involved in each I/O transfer. For the read test, each client instance +sequentially issued 16 4 KiB requests. To avoid interference, each request +accessed a unique file. \paragraph*{Write} -The client sends a request with a file name, size and payload. The server -will then create the file and write the payload. The server then sends an -acknowledgement to the client. - -The write test has clients each issue 16 requests sending 4 KiB of data to -be written to disk. Each client specifies a unique file name for each -request and all clients specify unique files from each other. The directory -containing all the files is deleted between each test iteration. +The write request is similar to the read request, except that the client +provides the data to be written to the file. The server receives the data, +creates the file, writes the data (using \code{O_DIRECT}), and finally reponds +to the client indicating the request completed successfully. As with the read +test, for our test, each client issued 16 requests each, with each request +containing 4 KiB of data. With each request, a unique file was created. \paragraph*{Read-null} -The read-null test is identical to the read request, except that the -server sends uninitialized data rather than performing any file I/O. The -server allocates a buffer for the response based on the size the client -requested. - -The read-null test had clients each issue 4096 requests requesting 4 KiB of -data to be returned from the server. +The read-null request is identical to a read request, except that the actual +I/O operation is not performed. Instead, the server sends a response +consisting of undefined data. For the read-null test, each client issued +4096 requests of 4 KiB each. \paragraph*{Write-null} -The write-null test is identical to the write request, except that the -server discards incoming data rather than performing any file I/O. The -server will still send an acknowledgement to the client. +Like the read-null request, the write-null request is the same as a write +request, except that the server skips the actual I/O write. Even though the +client includes the data with the request, it is discarded by the server. For +the write-null test, each client issued 4096 requests of 4 KiB each. -The write test had clients each issue 4096 requests sending 4 KiB of data to be -discarded by the server. - -\subsubsection{Experiment - Server Types} +%\subsubsection{Experiment - Server Types} We implemented the same request protocol in multiple server daemons in order to contrast different approaches to concurrent request processing. Each server uses the same fundamental coding style to the degree possible. One server is implemented using the \aesop language, while all other servers are implemented in C. The pthread library was -used in all cases that required explicit threading, while the +used where thread support was needed, while the libev~\cite{libev-web} library was used in all cases that required an -explicit event loop. +explicit event handling. A short summary of each server implementation follows +below. -Each server is setup to report the VmHWM stat when the application exits. +%Each server is setup to report the VmHWM stat when the application exits. \paragraph*{\aesop} -The \aesop server is implemented in the \aesop programming language. The server -uses a {\em lonely pbranch} to service each client. All operations for a client -are handled within a single pbranch. The socket and file operations are -performed with blocking \aesop functions that are provided by the \aesop -standard library. The underlying socket resource uses non-blocking sockets -with a thread pool of 12 threads. The file resource uses synchronous IO and -a thread pool with 4 threads. +The \aesop server is implemented using the \aesop programming language. A +{\em lonely pbranch} is used to service each client. All operations for a +client are handled within a single pbranch. The socket and file operations are +performed with blocking \aesop functions that are provided by the \aesop +standard library. On this system, the underlying socket resource uses +non-blocking sockets in an event driven model, with up to 12 threads driving +the event loops. The file resource uses synchronous I/O calls and a thread +pool with 4 threads. \paragraph*{Thread-per-client} -The thread-per-client server spawns a thread for each client connection. +The thread-per-client server spawns a dedicated thread for each client connection. All requests for a given client are handled within the same thread. This model -uses blocking socket operations and standard file read and write operations. -Each thread executes until the corresponding client disconnects. +uses synchronous socket operations and synchronous file read and write operations. +When the client disconnects, its associated thread is destroyed. \paragraph*{Thread-per-client-nb} The thread-per-client-nb server is identical to the thread-per-client -server, except that is uses non-blocking socket calls in place of blocking +server, except that is uses asynchronous socket calls in place of synchronous socket calls. For example, in order to send a message, a thread will -perform non-blocking sends until it encounters the EWOULDBLOCK error code. +perform asynchronous sends until it encounters the \code{EWOULDBLOCK} error code. It then polls the socket until it is ready and continues sending data. We implemented this version to investigate the possible performance difference between the synchronous and asynchronous socket calls in a scenario where -all other factors are held constant. +all other factors remain equal. \paragraph*{Thread-per-operation} -The thread-per-operation server uses an event loop to watch all sockets for -activity. When a new request is available, a thread is spawned and the request -is handled completely from within that thread. When the request is complete -the thread is destroyed. Blocking socket operations and standard file read -and write functions are used in this implementation. +The thread-per-operation server uses an event loop to watch all client +connections for activity. When a new request is available, a thread is +spawned and the request is handled completely from within that thread. When +the request is complete the thread is destroyed. Synchronous socket operations +and standard file read and write functions are used in this implementation. \paragraph*{Thread-pool} -The thread-pool server uses an event loop to watch all sockets for activity. -When a new request is available, the event loop puts the request on a queue and -wakes up a thread from the thread pool. The request is handled completely -from within a single thread of the thread pool. Blocking sockets and -standard file read and write functions are used in this implementation. The -thread pool was implemented following best practices for scalable condition -variable performance as described in \cite{hp-cond-variable}. +The thread-pool server is similar to the thread-per-operation server, in that +it uses an event loop to watch connected client connections for activity. +However, instead of creating a new thread to service each active client, the +connection is added to a queue. A fixed-size thread pool is used to service +the connections from the queue. The thread pool was implemented following +best practices for scalable condition variable performance as described in +\cite{hp-cond-variable}. + \paragraph*{Event} -The event server uses an event loop not only to detect incoming requests, +The event server uses an event loop not only to detect active connections, but to service them as well. Each request processing step is executed from -an event loop callback function. The event server uses non-blocking sockets -and asynchronous file I/O. Note that although this implementation does not -use any explicit threads, the operating system can still use multiple cores -to drive both the network and disk. - -\subsubsection{Experiment - Test System} - -All experiments were executed on the Fusion cluster managed by -the Argonne Laboratory Computing Resource Center -(LCRC). Fusion is a IBM iDataPlex dx360 M2 system. It features 320 compute -nodes which each consist of two Intel Nehalem 2.6 GHz Xeon processors and 36 GB -of RAM. The compute nodes have hyper threading disabled. The cluster has -an InfiniBand QDR interconnect. Each compute node also has a single SATA 7200 RPM -hard disk for local scratch storage. All disk IO was performed using the -local scratch storage and client/server communication was done over the -IB network using IPoIB. - -We executed 16 clients processes per phyiscal node. The tests were executed -on 65 nodes. One node was used for the server and the other 64 nodes were -used for the clients. We scaled the clients from 16 (1 node) to 1024 (64 nodes). +an event loop callback function. The event server uses asynchronous sockets +and asynchronous file I/O. Note that although this implementation does not use +explicitly create or use threads, the operating system can internally still +use multiple threads to drive both the network and disk. + +\par +All experiments were executed on the Fusion cluster managed by the Argonne +Laboratory Computing Resource Center (LCRC). Fusion is a IBM iDataPlex dx360 +M2 system. It features 320 compute nodes which each consist of two Intel +Nehalem 2.6 GHz Xeon processors and 36 GB of RAM. The compute nodes have hyper +threading disabled. The cluster has an InfiniBand QDR interconnect. Each +compute node also has a single SATA 7200 RPM hard disk for local scratch +storage. All disk IO was performed using the local scratch storage and +client/server communication was done over the IB network using IPoIB. We +instantiated 16 clients processes per phyiscal node. The tests were executed +on 65 nodes, one server node and 1 to 64 client nodes. \subsection{Runtime Efficiency} Figure~\ref{fig:write} shows the overall run time of the concurrent write workload for each server implementation as the number of client processes -is scaled from 16 to 1024. In this graph we see that \aesop does not +is scaled from 16 to 1024. In this graph we see that {\aesop} does not perform as well as the other servers for small workloads (delivering 79 -ops/second at the smallest scale, verses 132 ops/seconds for the thread-per-op -server). However, \aesop is the fastest server at the largest scale -(delivering 136 ops/second verses 125 ops/second for the nearest competitors +operations per second at the smallest scale, versus 132 ops/s seconds for the +thread-per-op server). However, {\aesop} is the fastest server at the largest +scale (136 ops/s versus 125 ops/s for the nearest competitors in thread-per-client and thread-per-client-nb). -Figure~\ref{fig:read} shows the results of the read experiment. -\aesop performs more favorably at small scale for this workload than in the +Figure~\ref{fig:read} shows the results of the read experiment. \aesop +performs more favorably at small scale for this workload than in the previously shown write workload. At the largest scale, \aesop completes the -test with 339 ops/sec verses 354 ops/sec for the fastest server (thread-pool), -which is a 4.5\% difference. -The event server performs particularly poorly in all cases, ultimately -running the largest scale test with only 212 ops/sec. +test with 339 operations per second versus 354 ops/s for the fastest server +(thread-pool), a 4.5\% difference. The event server performs +particularly poorly in all cases, ultimately running the largest scale test +with only 212 ops/s. The small scale results for \aesop may indicate that additional tuning is needed to improve latency for small test runs. The issue is likely @@ -236,19 +227,16 @@ fundamental programming language problem. \caption{Runtime performance for four test cases} \end{figure*} -\subsubsection{Latency} - -The client measures the latency of each individual request and then computes -the minimum and maximum latency, the first quartile latency and third +The test client also measures the latency of each individual request and then +computes the minimum and maximum latency, the first quartile latency and third quartile latency. The box represents the first and third quartiles and the -whiskers are the minimum and maximum values. The following results are for -the 1024 client size selected from the same iteration as the maximum runtime -graphs. Figure~\ref{fig:writelat} shows (in log scale) \aesop offers very -comparative latency performance with respect to the other configurations and -only notably -thread-per-op and event servers are significantly worse. The relative latency -metrics -are similar for the other test cases at 1024 clients. +whiskers are the minimum and maximum values. Figure~\ref{fig:writelat} shows +the request latency for 1024 clients in the case of the write test. From the +figure, it can be seen that the \aesop server implementation exhibits very +similar latency results when comparing to the other implementations. The +thread-per-op and event server variants perform significantly worse. As the +latency numbers for the other request types show similar results, only the +write test is shown. \begin{figure}[ht] \centering @@ -259,26 +247,29 @@ are similar for the other test cases at 1024 clients. \subsection{Memory Efficiency} Another aspect of the overall performance is the memory efficiency of each -server implementation. In this section we compare \aesop to the other server -implementations using the same runtime performance experitment. -The memory utilization of each server implementation was captured during -the runtime performance experiments. We recorded the VmHWM stat from the -server when the client test was completed. The VmHWM stat is a Linux-specific -metric that represents the peak resident set size (RSS) of an executable, -where RSS corresponds to the amount of paged-in memory used by the executable. - -In Figure~\ref{fig:writemem} (shown in log scale) we see that thread-pool limits the -memory usage as the client work load increases because the thread-pool -by design limits the number of requests that can be in progress at once. The -other server implementations scale as the number of clients increase. -Although the \aesop server cannot match the thread-pool server in terms of -memory usage, it does compare favorably to the thread-per-client and -thread-per-op servers. Note that the thread-per-client and thread-per-op -models consume virtual memory at a much larger rate due to the number of -thread stacks allocated. We chose not to evaluate this metric, however, -as the resident memory seems to be a more relevant metric in practice. Again, -the memory usage graphs for the other test cases are similar to those shown -in the write test case. +server implementation. In this section we compare the memory usage of the +\aesop server to that of the other server implementations. The memory +utilization of each server was captured during the runtime performance +experiments. We recorded the VmHWM stat from the server when the client test +was completed. The VmHWM stat is a Linux-specific metric that represents the +peak resident set size (RSS) of an executable, where RSS corresponds to the +amount of paged-in memory used by the executable. + +In Figure~\ref{fig:writemem} we see that thread-pool manages to limit the +memory usage even though the client work load increases. This can be +explained by the design of the thread-pool, which by limiting the number of +active threads used to service client connections also limits the number of +concurrent requests. For the other server implementations, memory usage +increases accordingly as the number of clients (and thus number of concurrent +requests) increases. Although the \aesop server cannot match the thread-pool +server in terms of memory usage, it does compare favorably to the +thread-per-client and thread-per-op servers. Note that the thread-per-client +and thread-per-op models consume virtual memory at a much larger rate due to +amount of virtual memory allocated to each thread stack. We chose not to +evaluate this metric, however, as the resident memory seems to be a more +relevant metric in practice. As the memory usage for remaining tests (read, +read-null and write-null) was very similar, only the write test results are +shown. \begin{figure}[ht] \centering @@ -288,12 +279,6 @@ in the write test case. \subsection{Productivity} -In order to evaluate our productivity claims, we first look at a quantitative -example. The completixy of each of the server implementations is compared -using standard measures. We then look at producitivty from a qualitiative -view considering the adaptation of a simple code example and anecdotal -evidence while running these tests. - \begin{table} \small \begin{center} @@ -302,7 +287,7 @@ evidence while running these tests. \hline & CC & mod. CC & SLOC \\ \hline -aesop & 16 & 11 & 179 \\ +\aesop & 16 & 11 & 179 \\ thread-per-client & 17 & 12 & 182 \\ thread-per-client-nb & 17 & 12 & 184 \\ thread-per-op & 22 & 17 & 249 \\ @@ -316,125 +301,133 @@ event & 28 & 23 & 341 \\ \vspace{-.2in} \end{table} -\subsubsection{Quantitative} +%\subsubsection{Quantitative} +We approximate programmer productivity by measuring code complexity. Table~\ref{tab:complexity} compares the code complexity of each server -implemenation using McCabe Cyclomatic Complexity (CC) \cite{mccabe}, +implemenation using McCabe Cyclomatic Complexity (CC)~\cite{mccabe}, Modified McCabe Cyclomatic Complexity (mod. CC), and Source Lines of Code (SLOC). The CC and Mod. CC metrics were measured -using the pmccabe tool, version 2.6, created by Paul Bame \cite{pmccabe}, +using the pmccabe tool, version 2.6, created by Paul Bame~\cite{pmccabe}, while the SLOC metrics were measured using the sloccount tool, -version 2.26, created by David A. Wheeler \cite{sloccount}. +version 2.26, created by David A. Wheeler~\cite{sloccount}. To simplify the comparison, all four servers had no error handling except for assertions on expected return codes. The protocol definition (ie, -request and acknowledgement structs) as well as helper functions to loop -over send and recv were not counted in any of the implementations, as these +request and acknowledgement structs) as well as helper functions for sending +and receiving were not counted in any of the implementations, as these were similar in all four. -\aesop and thread are very similar in terms of complexity, with the slight -increase in the thread model due to function calls needed to create and join -threads. +The \aesop and thread implementation are very similar in terms of complexity, +with the slight increase in the thread model due to function calls needed to +create and join threads. -Thread pool and event model are both much more complex than the thread or -\aesop model. An additional complexity of the event model which is not -captured by these metrics is the fact that control flow is not preserved -across the processing of a given request. For example, servicing a write -operation requires 5 disconnected event handlers. So although the event +The thread pool and event model implementation are both much more complex than +the thread or \aesop model. An additional complexity of the event model which +is not captured by these metrics is the fact that control flow is not +preserved across the processing of a given request. For example, servicing a +write operation requires 5 disconnected event handlers. So although the event model appears less complex according to CC and mod. CC, qualitatively it is significantly more challenging to develop. -\color{blue} -\subsubsection{Qualitative} +Another productivity aspect that is not well captured by the complexity +metrics is how easy it is to retarget our example server to a new architecture +or system. In our initial testing, we wanted to tune the \aesop server +implementation to perform well on a few test systems. During this testing we +tried different strategies (event based, threads and hybrid models) for +sending and receiving network data, we experimented with multiple threads +driving \aesop and we evaluated different I/O APIs. All of these changes were +done within the \aesop runtime library and never required changing the \aesop +server test code. This type of flexibility allows experimentation to +determine the best tuning strategy for any given system without having to +redesign the core software algorithms. + +%\subsubsection{Qualitative} \begin{comment} -The aesop code (listing~\ref{lst:aesop}) strongly resembles a threaded code. +The \aesop code (listing~\ref{lst:\aesop}) strongly resembles a threaded code. Instead of creating a thread, a lonely pbranch is used to make sure new connections can be accepted even though the previous connection is still being processed. To avoid name collisions, the \code{accept}, \code{read} and -\code{write} functions are prefixed, but this is not required by the aesop +\code{write} functions are prefixed, but this is not required by the \aesop language. The \code{handleConnection} function differs in that the function declaration is proceeded by the \code{__blocking} keyword. The \code{accept}, \code{read} and \code{write} functions are provided by the -aesop `socket' resource. While the aesop code, like a threaded version, -preserves linear control flow, the C code generated by the aesop compiler +\aesop `socket' resource. While the \aesop code, like a threaded version, +preserves linear control flow, the C code generated by the \aesop compiler actually resembles the event based code. As such, a single thread suffices to handle multiple connections concurrently. \end{comment} -The example discussed in this section is very simple and differs from real -world code by the absence of error handling, timeout handling and request -throttling. As an example, in \aesop a timeout can be easily implemented by adding a -\code{pbranch} which starts a timer which then cancels the original \code{pbranch} once -the timeout is reached. The original \code{handleConnection} pbranch upon -completion will cancel the timer pbranch so that the \code{pwait} block can -proceed. Listing~\ref{lst:aesop} shows a +The example server used for evaluation is very simple and differs from real +world code by the absence of error handling, time-out handling and request +throttling. As shown in section~\ref{sec:example}, \aesop provides powerful +primitives to simplify time-out and other error handling. The same cannot be +said of the event or threaded models, since neither model offers any help in +cancelling an outstanding operation. + +\begin{comment} +As an example, a timeout can be easily implemented by adding a +\code{pbranch} starting a timer which cancels the original \code{pbranch} once +the timeout is reached. The original \code{handleConnection pbranch} will +also cancel the timer when it completes. Listing~\ref{lst:aesop} shows a simple server design written in \aesop which uses a \code{lonely pbranch} to allow accepting new connections while the previous connection is being processed. The server processes a single request from the client and disconnects. Listing~\ref{lst:timeout} demonstrates how to add timeout -handling and cancellation to the client processing. -The same effect is not easily achieved +handling to the client processing. The same effect is not easily achieved using the event or threaded models, since neither model offers any help in cancelling an outstanding call. - -\begin{figure*}[ht] -\center -\begin{minipage}[b]{.45\textwidth} -\begin{lstlisting}[caption=Aesop code, label=lst:aesop] -// handling of client request -__blocking void handleConnection(int s) -{ - aesocket_read (s, request); - fd = aesop_open(); - aesop_read(fd, ack); - aesop_close(fd); - aesocket_write (s, ack); - aesocket_close(s); -} - -// server main loop -while (1) { - s = aesop_accept (sock); - pbranch { - handleConnection(s); - } -} -\end{lstlisting} -\end{minipage} -\quad -\begin{minipage}[b]{.45\textwidth} -\begin{lstlisting}[label=lst:timeout,caption=Example timeout handling in aesop] -// A simple wrapper around handleConnection -// demonstrates how to add a timeout using -// a timer and cancellation -__blocking void handleConnectionWithTO(int s, int timeout) -{ - pwait { - pbranch { - aesop_timer (timeout); - ae_cancel_branches (); - } - pbranch { - handleConnection(s); - ae_cancel_branches (); - } - } -} -\end{lstlisting} -\end{minipage} -\end{figure*} - -Another example we indirectly evaluted during our runtime analysis was the -flexibilty of changing the underlying concurrency models without modifing -the \aesop server source. In our initial testing, we wanted to tune \aesop -to perform well on a few test systems to estabilsh reasonable default settings. -During this testing we tried different strategies for the thread-pool -implementation, we experimented with multiple threads driving the event loop -and we using different file I/O APIs. All of these changes were done within -the \aesop runtime library and never required changing the \aesop server -test code. This type of flexibility allows experimentation to determine the -best tuning strategy for any given system without redesigning the core -software. - -\color{black} +\end{comment} +% +%\begin{figure*}[ht] +%\hfill +%\begin{minipage}[b]{.49\textwidth} +% \begin{lstlisting}[caption={aesop code}, label=lst:aesop] +%// handling of client request +%__blocking void handleConnection(int s) +%{ +% aesocket_read (s, request); +% fd = aesop_open(); +% aesop_read(fd, ack); +% aesop_close(fd); +% aesocket_write (s, ack); +% aesocket_close(s); +%} +% +%// server main loop +%while (1) { +% s = aesop_accept (sock); +% pbranch { +% handleConnection(s); +% } +%} +%\end{lstlisting} +%\end{minipage} +%\hfill +%\begin{minipage}[b]{.49\textwidth} +%\begin{lstlisting}[label=lst:timeout,caption=Example timeout handling in aesop] +%// A simple wrapper around handleConnection +%// demonstrates how to add a timeout using +%// a timer and cancellation +%__blocking void handleConnectionWithTO(int s, int timeout) +%{ +% pwait { +% pbranch { +% aesop_timer (timeout); +% ae_cancel_branches (); +% } +% pbranch { +% handleConnection(s); +% ae_cancel_branches (); +% } +% } +%} +%\end{lstlisting} +%\end{minipage} +%\hfill +%\end{figure*} +% +%\color{black} +%% +% diff --git a/papers/2011/aesop/implementation.tex b/papers/2011/aesop/implementation.tex new file mode 100644 index 0000000..53c3f78 --- /dev/null +++ b/papers/2011/aesop/implementation.tex @@ -0,0 +1,86 @@ +\label{section:implementation} + +We implemented the \aesop language using a source-to-source translator. It +processes \aesop code, and rewrites \aesop specific functionality using +standard C constructs, emitting fully compliant C code. Because of this, +\aesop code can be deployed on any system that provides a suitable C compiler. +As \aesop implements an extension to the C language, all valid C programs are +also valid \aesop programs and all C features can be used within an \aesop +program. Currently, the translator only needs to rewrite \code{__blocking} +functions; all other code is passed through unmodified. + +\begin{figure} + %\vskip -0.25in + \centering + \includegraphics[keepaspectratio,width=0.48\textwidth]{stack.pdf} + \caption{Model of Aesop usage. + \label{fig:model} + } +\end{figure} + +Figure~\ref{fig:model} shows how a blocking function \code{service_function} +is translated by the current version of the \aesop translator. As a first +step, all blocking function calls within the function body are located. Based +on the location of the blocking calls, the function is effectively split in a +set of code blocks (states), where each code block only contains non-blocking +(traditional C) function calls. The blocking calls themselves are rewritten in +an asynchronous form. For example, \code{network_operation} becomes +\code{post_network_operation}. This post function becomes the last statement +of the code block, where the first statement following the +\code{network_operation} call becomes the first statement of the next block. +As part of the \code{post_network_operation} call, \aesop provides a callback +argument which indicates where execution needs to continue once the operation +completes. As soon as the operation is posted, \code{service_function} +returns, unwinding the stack. Depending on the state and structure of the +program, the stack might unwind all the way down to the internal \aesop +function which initially called \code{aesop_main}, the \code{__blocking} +starting point of an \aesop program. When this happens, \aesop will try to +make progress on any pending operation, until one or more operations complete. +As part of the completion, the callback (passed to the post function, for +example \code{post_network_operation}) will be invoked, effectively resuming +execution with the statement following the \code{__blocking} call. + +Our current translator does not actually split a blocking function into +multiple independent functions. Instead, the function signature is modified +(to add the context and other internal parameters), and a jump table is +generated at the beginning of the function. The jump table uses the +\code{goto} instruction to jump to a specific label within the function. +Labels are inserted at the beginning of each internal state (typically the +beginning of the function, and every statement following a blocking call). +These modifications make it possible to continue the function at the appropriate +location once its execution can be resumed. + +As an optimization, \aesop recognizes when an operation completes +before the post function returns. When this happens, no stack +unwinding occurs and execution immediately continues with the next state +in the function. + +% variables, state management +Because variables local to the function scope are stored on the stack, and are +destroyed when returning from the function, \aesop\ -- as part of the +rewriting process -- moves all function scope variables (and function +arguments) into a context structure on the heap, transparently rewriting +variable references to account for this change. A pointer to this context is +passed to the callback when the execution of the function resumes. + +% cancel +Within a given context, \aesop keeps track of pending operations. When a +request is made to cancel concurrent branches, \aesop will call the correct +cancellation function for each pending operation in the current context. To do +so, \aesop needs to know, for each blocking call, how to cancel operation. +For blocking functions translated by \aesop (those marked by +\code{__blocking}), \aesop internally provides the cancel methods. +However, for operations managed by an existing, external library (for example +AIO), this information needs to be provided to \aesop. This is done by +creating a {\em resource}. \aesop resources enable implementing an \aesop +blocking function using pure C. Resources are typically used to provide an +\aesop binding on top of an existing C library. A resource exports an \aesop +API consisting of regular and blocking functions, and internally registers a +set of hooks which can be called by \aesop to make progress, test or cancel a +blocking function exported by the resource API. Progress can be made by +providing a polling function, which will be called by \aesop when no other +work is available. Alternatively, at any time, a resource can asynchronously +request for its polling function to be called. However, \aesop does not +enforce a progress model on the resource. Some resources do not define a +polling function, typically because the resource or the underlying C library +can make progress independently. diff --git a/papers/2011/aesop/intro.tex b/papers/2011/aesop/intro.tex index 8e7df47..9f33301 100644 --- a/papers/2011/aesop/intro.tex +++ b/papers/2011/aesop/intro.tex @@ -1,11 +1,13 @@ %% Describe complex path of file system operations to servers +\label{sec:intro} + HPC and distributed systems rely on a diverse collection of system software to provide shared services for applications. Examples include file storage, scheduling, security, checkpointing, web services, membership management, and fault detection. Today's systems demand that these services scale to hundreds of thousands of concurrent clients, -with millions of clients expected in the near future. {\color{red}KH: need citation. Reference to Table~\ref{tab:counts}} Developing system +with millions of clients expected in the near future. Developing system software for this environment is a complex task due to a number of competing software engineering requirements: @@ -14,104 +16,94 @@ software engineering requirements: \item maintainable code for complex distributed algorithms, \item portability across diverse architectures, \item management of concurrent operations across multiple hardware devices, -\item and maximum performance +\item and maximum performance on multiple platforms \end{itemize} -Further complicating matters is the fact that system software services -must not only interact with multiple hardware devices concurrently, -but do so using a wide assortment of interfaces and progress models. -Popular network transports, disks, and local databases may present a -simple set of blocking functions, fully asynchronous callbacks, or any one -of a number of API models in between. This diversity makes it difficult to -build algorithms that are portable and yet capable of leveraging each -component in an optimal manner. +Further complicating matters is the fact that system software services must +not only interact with multiple hardware devices concurrently, but do so using +a wide assortment of interfaces and progress models. Popular network +transports, disks, and local databases may present a simple set of blocking +functions, fully asynchronous callbacks, or any one of a number of API models +in between. In some cases, the most efficient API is platform specific. This +diversity makes it difficult to build algorithms that are portable and yet +capable of leveraging each component in an optimal manner. -The two most common models for managing asynchronous activity -are multithreading and event-driven architectures. -Examples of both can be found in a variety of large scale -production services. The Apache web server advocates a -multithreaded mode of operation for high performance -deployments~\cite{apache-threads}. -In contrast, the memcached object caching service uses an event-driven -model with threads used to drive concurrent event +The two most common models for managing asynchronous activity are +multithreading and event-driven architectures. Examples of both can be found +in a variety of large scale production services. The Apache web server +advocates a multithreaded mode of operation for high performance +deployments~\cite{apache-threads}. In contrast, the memcached object caching +service uses an event-driven model with threads used to drive concurrent event loops~\cite{memcached-threads}. -Multithreading is popular for several reasons. First and -foremost, this model is widely -used and therefore well understood in the development community. -It produces a natural control flow in algorithm implementations -because each thread executes sequentially and issues simple blocking -device operations. Thread scheduling and stack management is handled by the -operating system. Threads also provide a clear model -to express concurrency. There are a number of challenges to building -a multi-threaded service, however. The first is that it is not always -intuitive how to best provision thread resources; threads can be assigned -per client, per request, or per underlying device operation, and they -may be allocated on demand or from an existing pool. Each technique or -combination of techniques presents tradeoffs in scalability, performance, -and readability, yet it is difficult to transition between them -during the development process. The second challenge is the disconnect -between the number of threads needed to express -logical concurrency and the number of threads needed -to make optimal use of local resources. For example, even if there -are 1,000 client requests in flight at once, the server may have only a few -processing cores and a storage device that can only effectively handle a few -operations at a time. This disconnect can cause a seemingly simple thread -model to evolve into a more complex framework in order to take local resource -scheduling constraints into account. +Multithreading is popular for several reasons. First and foremost, this model +is widely used and therefore well understood in the development community. It +produces a natural control flow in algorithm implementations because each +thread executes sequentially and issues simple blocking device operations. +Thread scheduling and stack management is handled by the operating system. +Threads also provide a clear model to express concurrency. There are a number +of challenges to building a multi-threaded service, however. The first is +that it is not always intuitive how to best provision thread resources; For +network services, threads can be assigned per client, per request, or per +underlying device operation, and they may be created on demand or allocated +from an existing pool. Each technique or combination of techniques presents +tradeoffs in scalability, performance, and readability. Yet it is difficult to +transition between them during the development process. The second challenge +is the disconnect between the number of threads needed to express logical +concurrency and the number of threads needed to make optimal use of local +resources. For example, even if there are 1,000 client requests in flight at +once, the server may have only a few processing cores and a storage device +that can only effectively handle a few operations at a time. This disconnect +can cause a seemingly simple thread model to evolve into a more complex +framework when taking local resource scheduling constraints into account. -Event-driven models can be used to address some of the common -challenges in thread efficiency. Event-driven models strive to unify -the management of asynchronous activities, so that the completion of -any storage or network event triggers execution of the next servicing -step through a centralized event handler. This architecture provides -greater control over asynchronous activity and avoids -the use of operating system threads to express logical concurrency. Any -number of client requests can be tracked and serviced without explicitly -allocating a thread to each request. There are drawbacks to this model as -well, however. -Event-driven architectures break the logical control flow of software -algorithms into into disjoint segments at each point where the algorithm -interacts with an asynchronous device or operating system service. -Per-request state has to be managed manually by the programmer. In effect, -it turns the single linear control flow for an algorithm into a state -machine with several independent states. Because these states are invoked -from a centralized event handler, their interfaces must be consistent and -generalized. -This makes it difficult to verify dependencies or perform type checking -between states. Pure event-driven models also do not directly address the -requirement of multi-core architectures or high performance peripherals that -require multiple threads to achieve maximize throughput. Event-driven servers -therefore tend to evolve during the development process to take on -characteristics of both event-driven and multithreaded services in order to -compensate for these issues. +Event-driven models can be used to address some of the common challenges in +thread efficiency. Event-driven models strive to unify the management of +asynchronous activities, so that the completion of any storage or network +event triggers execution of the next servicing step through a centralized +event handler. This architecture provides greater control over asynchronous +activity and avoids the use of operating system threads to express logical +concurrency. Any number of client requests can be tracked and serviced without +explicitly allocating a thread to each request. There are drawbacks to this +model as well, however. Event-driven architectures break the logical control +flow of software algorithms into into disjoint segments at each point where +the algorithm interacts with an asynchronous device or operating system +service. In effect, it turns the single linear control flow for an algorithm +into a state machine with several independent states. Because these states +are invoked from a centralized event handler, their interfaces must be +consistent and generalized. This makes it difficult to verify dependencies or +perform type checking between states. Per-request state has to be managed +manually by the programmer, often referred to as {\em stack +ripping}~\cite{krohn2007ecm}. Pure event-driven models also do not directly +address the requirement of multi-core architectures or high performance +peripherals that require multiple threads to achieve maximize throughput. +Event-driven servers therefore tend to evolve during the development process +to take on characteristics of both event-driven and multithreaded services in +order to compensate for these issues. -In general there is no clear-cut metric that can be used to identify the -best concurrent programming model for a given system software project, -yet this decision affects every aspect of the design from algorithm -implementation to device management. In order to help address this -problem, we propose a new programming language and programming model, -\aesop, that targets the software engineering challenges associated -with developing distributed system software. It's goal is to improve -programmer productivity and architecture flexibility without compromising -the efficiency of large-scale system software. +In general there is no clear-cut metric that can be used to identify the best +concurrent programming model for a given system software project, yet this +decision affects every aspect of the design from algorithm implementation to +device management. In order to help address this problem, we propose a new +programming language and programming model, \aesop, that targets the software +engineering challenges associated with developing distributed system software. +Its goal is to improve programmer productivity and architecture flexibility +without compromising the execution efficiency. -\aesop is a superset of the C language. It describes blocks of code that -should be executed asynchronously or concurrently, without dictating -whether those routines must be implemented with threads or not, and -without dictating the asynchronous progress model to be used by any -underlying devices. Because the core algorithm description is decoupled -from these architecture-specific details, the run-time system can be -tuned to match different system architectures, different device APIs, -and different workloads without any change to the core system software -algorithms. Unlike event-driven architectures, the \aesop extensions -also preserve readable control flow regardless of how many steps or +\aesop is a superset of the C language. It describes blocks of code that can +be executed asynchronously or concurrently, without dictating whether those +blocks must be implemented with threads or not, and without dictating the +asynchronous progress model to be used by any underlying devices. Because the +core algorithm description is decoupled from these architecture-specific +details, the run-time system can be tuned to match different system +architectures, different device APIs, and different workloads without any +change to the core system software algorithms. Unlike event-driven +architectures, the \aesop extensions do not suffer from an inversion of +control, preserving readable control flow regardless of how many steps or concurrent devices are utilized in a given algorithm. \aesop also offers -functionality that is simply not available in a traditional multithreaded or -event-driven framework, \todo{private/variable: not true. TLS} -including the ability to mark variables as -shared or private across concurrent execution paths as well as a robust -mechanism to cleanly cancel concurrent execution paths. +functionality that is not available in a traditional multithreaded or +event-driven framework, for example the ability to cleanly cancel outstanding +operations. \subsection{Summary of contributions} @@ -122,7 +114,8 @@ as \aesop, for use in the development of distributed system software \item The definition of new language constructs to aid managing highly concurrent execution \item An case study comparing \aesop to five popular multithreaded -and event-driven architectures for a simple distributed service +and event-driven architectures, using a simple distributed service as an +example \item A quantitative analysis of the code complexity of the architectures used in the case study \end{itemize} @@ -130,10 +123,10 @@ used in the case study The remainder of this paper is organized as follows. Sections~\ref{section:context} and~\ref{section:related} describe the challenges of distributed system software development and related work in -addressing those challenges. Section~\ref{section:model} -presents the \aesop programming model and -programming language. Section~\ref{section:evaluation} describes a simple -example distributed service and uses it as a case study to contrast \aesop +addressing those challenges. Sections~\ref{section:model} +and~\ref{section:implementation} present the \aesop programming model and +our implementation thereof. Section~\ref{section:evaluation} describes a simple +distributed service and uses it as a case study to contrast \aesop against other models in terms of performance, memory efficiency, and productivity. Section~\ref{section:conclusion} summarizes our findings and proposes avenues of future work. diff --git a/papers/2011/aesop/model.tex b/papers/2011/aesop/model.tex index 62f02ff..ab61b63 100644 --- a/papers/2011/aesop/model.tex +++ b/papers/2011/aesop/model.tex @@ -7,29 +7,24 @@ program model. Concurrency is expressed in \aesop through the use of {\em parallel branches} or {\em pbranches}. A parallel branch groups a list of statements and -supports branch-scoped variables, much like a regular function. {\color{red}However, when -multiple which will be executed sequentially, as in a normal C program.} +supports branch-scoped variables, much like a regular function. Within a +pbranch, statements are executed sequentially, as in a normal C program. However, when multiple pbranches are active, the \aesop language only enforces -sequential execution on the scope of a single pbranch. Statements from other -pbranches might interleave execution or might execute concurrently (using a -thread for example). +sequential execution within the scope of a single pbranch. Statements from +other pbranches might interleave execution or might execute concurrently +(using a thread for example). -It is possible to synchronize with pbranches using the {\em pwait} construct. +It is possible to synchronize with pbranches using the \code{pwait} construct. A pwait is similar to the barrier implicit in many OpenMP directives (such as parallel for), in that no statements following the pwait will execute unless all of the pbranches it encloses have completed. -By default, variables in \aesop are shared between pbranches. Marking a -variable as {\em pprivate} gives each pbranch a private shadow copy, initialized -using the shared instance when the branch is created. \color{red}PHC: -clarify that when you say variables are shared between pbranches by default, -you are only talking about variables declared in the scope containing the -pbranch. Variables actually declared in the pbranch itself are private by -default.\color{black} - -\color{red}PHC: also, if there is a similar notation in any of the other -parallel programming models (openmp or whatever) for shared/private -variables then we should cite at least one example here.\color{black} +By default, variables in \aesop are shared between pbranches. This is similar +to OpenMP, where variables are shared between multiple threads unless +indicated otherwise. Marking a variable as \code{pprivate} gives each pbranch +a private shadow copy, initialized using the shared instance when the branch +is created. Variables declared within a pbranch are -- adhering to the C +scoping rules -- necessarily private. It is possible for a pbranch to request the termination (or cancellation) of all other pbranches within the same pwait by calling the @@ -37,17 +32,43 @@ all other pbranches within the same pwait by calling the other pbranches have been marked for cancellation; it is asynchronous in that it does not wait for the other pbranches to exit. While pbranches are not forcibly terminated, a pbranch with an outstanding cancellation request -exhibits a slightly different behavior to ensure a race free delivery of the -cancellation signal. By design, it is not possible for a pbranch to clear its -cancellation flag. \color{red}PHC: we probably have to clarify "slightly -different behavior" there to keep reviewers from flipping out. Maybe just -say that a cancelled pbranch will receive a well-known (ECANCELLED) error -code from affected functions, and that a pbranch can also check its -cancellation state manually at any time?\color{black} - -\color{red}PHC: it might also be good to contrast the aesop cancel -functionality with pthread\_cancel() and point out why the aesop -functionality is better.\color{black} +affects branch execution to ensure a race free delivery of the cancellation +signal. If the pbranch is in a function call when the cancellation signal +arrives, that function will be notified of the cancellation, enabling the +function to take whatever steps necessary to cancel the operation and return a +suitable return code. Further function calls within the cancelled pbranch will +detect the cancellation state upon entering the function and react +accordingly. By design, it is not possible for a pbranch to clear its +cancellation flag. + +%\todo{Probably Need to move the following two paragraphs to the +%introduction/related work} + +\aesop's cancellation model differs from others in that the language itself +takes care of cleanly cancelling pending operations, transparently emitting +the needed code to do so. For example, if a device exposes an asynchronous +API, \aesop will automatically call the corresponding \code{cancel} function +when a pbranch waiting for the completion of an operation is cancelled. +Many other libraries aimed at concurrent or asynchronous execution do not +support cancellation or only offer limited support. +In Grand Central Dispatch, once a code block is dispatched to a queue, +it can no longer be removed or cancelled. Likewise, OpenMP does not support +cancelling helper threads in parallel for and other concurrency constructs. +In both cases, the recommended practice is for the programmer to repeatedly +check a cancellation flag. This becomes especially cumbersome if other +functions are called from within the concurrent code, as now those functions +have to have access to the cancellation variable as well. + +While the POSIX thread API supports a number of different cancellation modes, +using them in production software is not trivial. In the \code{asynchronous} +cancellation mode, depending on the system, thread execution might be halted +at any point, making it very hard for the programmer to properly track cleanup +state. In the default \code{deferred} mode, cancellation is +ignored by all but a small list of special functions (termed +\code{cancellation points}). Many common C functions, such as \code{fread}, +may or may not -- depending on the system -- support cancellation. Most +don't support cancellation at all. This means that a portable program cannot +call any of these functions if timely cancellation is required. \aesop also supports {\em lonely pbranches}, i.e.\ pbranches created outside of a pwait. A lonely pbranch is similar to a POSIX detached thread in that @@ -64,18 +85,21 @@ regular C functions in that only blocking functions can contain pbranches. In addition, blocking functions support {\em cancellation}. Functions can be made blocking by adding the \code{__blocking} -qualifyer to the function declaration. However, blocking functions can only be +qualifier to the function declaration. However, blocking functions can only be called (either directly or indirectly through a blocking function pointer) by other blocking functions. This effectively requires the starting function of -an \aesop program to be blocking. \color{red}PHC: what about bindings like -we use for the key/val API? This is perhaps important to show that Aesop -can be used not only on the server side, but also in libraries that are -linked to standard applications.\color{black}. - +an \aesop program to be blocking. \aesop libraries form an exception to this +rule. To simplify calling \aesop libraries from C +and other languages, it is possible to generate C function stubs for a +blocking \aesop function. These stubs can either be synchronous, in which case +the stub only returns once the blocking function completed, or asynchronous, +in which case the stub returns as soon as progress can no longer be made. +To handle completion in the latter case, a callback -- called when the +blocking function completes -- can be specified. % cancellation -A active blocking function in a pbranch receiving a cancellation request will +An active blocking function in a pbranch receiving a cancellation request will be notified of the cancellation. The effect of this notification, and the way the function reacts to this request, depends on the implementation of that particular function. Likewise, any subsequent blocking call initiated in that @@ -106,76 +130,76 @@ In effect, the \aesop programming model provides for multiplexing multiple pbranches onto a single thread or core, similar to application implemented user space threading with voluntary yielding. This does not preclude true multi-threading; \aesop is fully thread-safe, and it is possible to have -multiple OS threads executing within a single aesop program. In fact, to fully -exploit multi-core systems, we routinely use multiple threads to run aesop +multiple OS threads executing within a single aesop program. In fact, +we routinely use multiple threads to run aesop code. In that case, if more than one pbranch becomes ready for execution, multiple threads will be used to execute those branches concurrently. -\color{red}PHC: clarify that when you say "we routinely use multiple threads -to run aesop code" you don't mean that the developer is explicitly -putting threads in their code; you are talking about something that's -happening within the aesop runtime engine.\color{black} - - -\subsection{Example Aesop Code} \begin{figure} - %\vskip -0.25in - \centering - \includegraphics[keepaspectratio,width=0.48\textwidth]{stack.pdf} - \caption{Model of Aesop usage. - \label{fig:model} - } +\center + \begin{lstlisting}[caption={Aesop Code Example}, + label=lst:parallelfor] +__blocking void doReplicaWrite (...) { + pwait { + pbranch { concurrent_step_1 (); } + pbranch { concurrent_step_2 (); } + pbranch { concurrent_step_3 (); } + } +} +__blocking void replicaWrite (...) { + pwait { + pbranch { + aesop_timer (TIMEOUT); + ae_cancel_branches (); + } + pbranch { + doReplicateWrite (...); + ae_cancel_branches (); + } + } +} +__blocking void write (...) { + pwait { + pprivate int i; + for (i=0; i<REPLICAS; ++i) + pbranch { + replicaWrite (i, ...); + } + } +} +\end{lstlisting} \end{figure} -Figure~\ref{fig:example} shows a common aesop code pattern. + +\subsection{Example Aesop Code} +\label{sec:example} +Listing~\ref{lst:parallelfor} shows a common aesop code pattern. Network servers, having a finite set of resources, have to protect against unbounded resource consumption by misbehaving or failing peers. This is typically done by placing an upper limit on the time resources are dedicated to a request, cancelling the request when the allocated time is exceeded. - -%% \begin{figure} -%% \begin{lstlisting} -%% __blocking void doWrite (...) { -%% pwait { -%% pbranch { -%% aesop_timer (TIMEOUT); -%% ae_cancel_branches (); -%% } -%% pbranch { -%% replicateWrite (...); -%% ae_cancel_branches (); -%% } -%% } -%% } -%% __blocking void replicateWrite (...) { -%% pwait { -%% pprivate int i; -%% for (i=0; i<REPLICAS; ++i) -%% pbranch { -%% replicateWriteTo (i); -%% } -%% } -%% } -%% \end{lstlisting} -%% \caption{Aesop Code Example} -%% \end{figure} - -\begin{figure} -\input listings/listing-1.ae.tex -\caption{Aesop example code.} -\label{fig:example} -\end{figure} - -\color{red} -Somewhere (not sure where in the paper) we should address how aesop uses -multiple cores. Right now, normal c code may or may not be executed in a -thread depending one what path is the lowest latency following a given -resource event. However, you can add advisory hints to tell aesop to run -code branches in a thread? See discussion on mailing list and go with -whatever we end up settling on there. -\color{black} - +In the code example, \code{write} concurrently forwards the write operation to +\code{REPLICAS} replicas. This is done using a for loop containing a pbranch. +Since the pbranch statement does not wait for the completion of the branch, +the next iteration of the \code{for} loop can start before the previous one +completed. In \code{replicateWrite}, a common pattern for easily handling +time-outs can be seen. Two pbranches are created, one to execute the work and +the other one to bound maximum time the other branch is allowed to execute. +When one of the branches completes, as a last task it cancels the other +branch. This means that if \code{doReplicateWrite} (line 15) completes first, +the operation completed within the allocated time frame, and the timer (line +4) will be cancelled. However, if line 15 takes longer than \code{TIMEOUT} +milliseconds, the timer (line 11) will complete first causing the ongoing +\code{doReplicaWrite} operation to be cancelled. Note how \code{replicaWrite}, +in a straightforward an reusable manner, adds a time-out to the existing +\code{doReplicaWrite} call. + +The short code fragment from listing~\ref{lst:parallelfor} effectively +implemented the pattern shown in figure~\ref{fig:sm}, in which a number of +concurrent write operations are started, each consisting of three tasks which +can be executed concurrently and each write operation is protected by a time +out. diff --git a/papers/2011/aesop/related.tex b/papers/2011/aesop/related.tex index 5d4cee5..b7fec39 100644 --- a/papers/2011/aesop/related.tex +++ b/papers/2011/aesop/related.tex @@ -2,48 +2,99 @@ \section{Related Work} \label{section:related} -A number of programming languages and programming models have been -developed to express application-level concurrency and efficiently leverage -multi-core architectures. Examples include -OpenMP~\cite{Chandra:2001:PPO:355074}, -X10~\cite{Charles:2005:XOA:1094811.1094852}, Unified -Parallel C~\cite{upc-intro}, -Chapel~\cite{Chamberlain:2007:PPC:1286120.1286123}, +The \aesop language described in this work builds upon lessons learned from +the event-driven state machine model used in the PVFS file +system~\cite{pvfs-web}. In PVFS State Machines (SM), every client and server +operation is expressed as a state machine using a language that is based on C +with extensions to describe service states and the transitions that link them. +A state is simply a C function. Each state function can optionally end by +submitting an asynchronous network, disk, timer, or pipeline operation through +an API that provides unified notification of completion events for each +device. When one of these operations completes, a state machine engine maps +the operation back to the appropriate state machine and executes the next +state function depending upon the outcome of the asynchronous operation. +Multiple state machines instances can be active simultaneously, allowing PVFS +to track the state of an arbitrary number of concurrent requests and make +progress on them without the overhead of explicit threading. + +This PVFS state machine model is essentially a formal framework for +event-driven programming. This approach has proved successful in achieving +high performance in large-scale production HPC +environments~\cite{pvfs-bgp-sc09}, but it shares many of the same developer +productivity challenges found in other event-driven models. First, all +algorithms must be split into functions on boundaries according to the +location of asynchronous operations, rather than according to logical +functionality. Secondly, the state machine engine uses a fixed function +prototype for each state machine function, meaning that arguments are passed +between state functions via an opaque struct that is shared across the entire +state machine. This makes automatic depencency checking between states +impossible (e.g., did the previous state function set all of the struct fields +needed by the current state function?) and leads to a new class of bugs that +do not typically appear in traditional C code. Additionally, PVFS SMs perform +all non-asynchronous computation on a single core, which is an undesirable +model given the advent of many-core servers. Finally, PVFS SM code does not +resemble traditional C code, making it difficult to reason about many +development challenges. + +These drawbacks introduce a learning curve for researchers and a additional +maintenance workload for maintainers. In this work, we seek to learn from the +lessons of PVFS SM development and present a programming model that provides +at least as much technical functionality but also allows file system +developers to write algorithms using a more traditional code organization. + +A number of programming languages and programming models have been developed +to express application-level concurrency and efficiently leverage multi-core +architectures. Examples include OpenMP~\cite{Chandra:2001:PPO:355074}, +X10~\cite{Charles:2005:XOA:1094811.1094852}, Unified Parallel +C~\cite{upc-intro}, Chapel~\cite{Chamberlain:2007:PPC:1286120.1286123}, Fortress~\cite{fortress}, CUDA~\cite{cuda-web}, and OpenCL~\cite{opencl08}. -Such languages allow developers to annotate code -blocks that should execute in parallel, including constructs for -synchronization and sharing variables. While many of these constructs -share principles with \aesop, they were developed with the goal of -coordinating computation on multiple cores at the application level, -rather than coordinating asynchronous activity on multiple I/O devices -at the system software level. As a result these languages optimize for -maximum use of available CPU resources, while \aesop instead optimizes for -maximum use of underlying devices that may or may not require significant -CPU activity. The level of \aesop concurrency is not strongly correlated -with the number of CPU cores available, though it does utilize multiple -cores (if available) to improve performance. - -Grand Central Dispatch (GCD)~\cite{gcd-web} is a popular programming -model introduced by Apple Inc. that allows developers to describe -tasks to be executed concurrently. GCD manages an implicit thread -pool to schedule and execute the tasks. If the tasks are CPU bound, -then GCD may match the number of threads to the number of CPU cores. -If the tasks block on device activity, then it may instantiate a larger -number of threads. The primary advantages of GCD are that it simplifies -multithreaded programming and automatically adjusts concurrency according -to the workload and the system architecture. Unlike \aesop, however, GCD -always uses threads to achieve concurrency and provides no general framework for -integration with device API's that provide more scalable asynchronous interfaces. - -In addition to programming models and languages, a number of support -libraries have also been developed to aid in portably managing a large -number of asynchronous events. Examples include libevent~\cite{libevent-web} -and libev~\cite{libev-web}. Such libraries provide an efficient unified -API for managing events, particularly on sockets or file descriptors. -While such libraries solve an important technical problem related to -service efficiency, they do not address the programmability aspect of -event-driven concurrent programming. It is up to the developer to fit -his algorithm logic to the basic event-driven model. +Such languages allow developers to annotate code blocks that should execute in +parallel, including constructs for synchronization and sharing variables. +While many of these constructs share principles with \aesop, they were +developed with the goal of coordinating computation on multiple cores at the +application level, rather than coordinating asynchronous activity on multiple +I/O devices at the system software level. As a result these languages +optimize for maximum use of available CPU resources, while \aesop instead +optimizes for maximum use of underlying devices that may or may not require +significant CPU activity. The level of \aesop concurrency is not +correlated with the number of CPU cores available, though \aesop can utilize +multiple cores (if available) to improve latency and concurrency. + +% this is repeated from above +\aesop has clear similarities to OpenMP, the C/Fortran language extensions for +expressing thread parallelism. In particular \aesop supports decoupling +sequential control flow (i.e. parallel for), waiting for their completion +(barrier) and the annotation of variables to indicate if they are to be shared +or private with respect to concurrent execution flows. As all these concepts +are present in OpenMP as well, it's easy to see how \aesop might be used to +write multithreaded applications. However, this is not the primary focus of +\aesop. \aesop does not create or require threads. While OpenMP mainly +provides a portable thread abstraction, it does not offer any support for +asynchronous device interfaces. \aesop also differs from the related work in +that only \aesop offers language support for cleanly cancelling outstanding +concurrent execution paths. + +Grand Central Dispatch (GCD)~\cite{gcd-web} is a popular programming model +introduced by Apple Inc. that allows developers to describe tasks to be +executed concurrently. GCD manages an implicit thread pool to schedule and +execute the tasks. If the tasks are CPU bound, then GCD may match the number +of threads to the number of CPU cores. If the tasks block on device activity, +then it may instantiate a larger number of threads. The primary advantages of +GCD are that it simplifies multithreaded programming and automatically adjusts +concurrency according to the workload and the system architecture. Unlike +\aesop, however, GCD always uses threads to achieve concurrency and provides +no general framework for integration with device API's that provide more +scalable asynchronous interfaces. GCD does not support cancellation of tasks. + +In addition to programming models and languages, a number of support libraries +have also been developed to aid in portably managing a large number of +asynchronous events. Examples include libevent~\cite{libevent-web} and +libev~\cite{libev-web}. Such libraries provide an efficient unified API for +managing multiple kinds of events, particularly on sockets or file +descriptors. While such libraries solve an important technical problem +related to service efficiency, they do not address the programmability aspect +of event-driven concurrent programming. It is up to the developer to fit his +algorithm logic to the basic event-driven model. % *Research* work on expressiveness of concurrency: % recognized a problem with expressiveness for a long time @@ -75,15 +126,3 @@ his algorithm logic to the basic event-driven model. % data parallelism (symmetric) - same operation - different data elements, MPI_Allreduce % task parallelism (asymmetric) - different operations (tasks) -% this is repeated from above -\aesop has clear similarities to OpenMP, the C/Fortran language extensions for -expressing thread parallelism. In particular \aesop supports decoupling -sequential control flow (i.e. parallel for), waiting for their completion -(barrier) and the annotation of variables to indicate if they are to be shared -or private with respect to concurrent execution flows. As all these concepts -are present in OpenMP as well, it's easy to see how \aesop might be used to -write multithreaded applications. However, this is not the primary focus of -\aesop. While OpenMP mainly provides a portable thread abstraction, it does not -offer any support for asynchronous device interfaces. -\aesop also differs from the related work in that only \aesop offers language -support for cleanly cancelling outstanding concurrent execution paths. hooks/post-receive -- Triton-private Repository

1 0

Triton Repository branch, master, updated. 2bc967ea2524a1a93cc596863cee1e156e619afe
by noreply＠mcs.anl.gov 13 Mar '12

13 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via 2bc967ea2524a1a93cc596863cee1e156e619afe (commit) via 0529715ef9f2567be8e607cc6e187927121bb985 (commit) via 325f4bdd8b58fa394e90595829f3563b1d8fc173 (commit) from 48125e56f22f8ec5fc7698c6e174194e70c461f4 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 2bc967ea2524a1a93cc596863cee1e156e619afe Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Mar 13 11:46:12 2012 -0500 Fix node number as string issue; progress on rosd-enqueue commit 0529715ef9f2567be8e607cc6e187927121bb985 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Mar 13 11:40:48 2012 -0500 Drop rosd-test-main commit 325f4bdd8b58fa394e90595829f3563b1d8fc173 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Mar 6 09:57:40 2012 -0600 Duplicate test rosd-injection and rosd-enqueue ----------------------------------------------------------------------- Summary of changes: code/src/fakess/fakess.ae | 28 ++--------- code/src/fakess/fakess.hae | 2 - code/src/mapping/mapping.ae | 6 ++- code/src/rebuild/rebuild.ae | 3 + code/src/replicated-osd/tests/module.mk.in | 9 ---- code/src/state/state.ae | 11 ++++- code/src/state/state.hae | 2 - code/src/state/tests/module.mk.in | 19 ++++---- .../tests/{rosd-injection.ae => rosd-enqueue.ae} | 50 ++++++++++++++++---- .../tests/{rosd-injection.sh => rosd-enqueue.sh} | 10 ++-- code/src/state/tests/rosd-injection.sh | 2 + 11 files changed, 78 insertions(+), 64 deletions(-) copy code/src/state/tests/{rosd-injection.ae => rosd-enqueue.ae} (67%) copy code/src/state/tests/{rosd-injection.sh => rosd-enqueue.sh} (67%) Diff of changes: diff --git a/code/src/fakess/fakess.ae b/code/src/fakess/fakess.ae index daea3cd..c2097e8 100644 --- a/code/src/fakess/fakess.ae +++ b/code/src/fakess/fakess.ae @@ -189,25 +189,9 @@ triton_ret_t triton_fakess_init_stage2(void) return result; } -__blocking triton_ret_t triton_fakess_init_self(void) -{ - triton_ret_t result; - triton_node_t self; - - assert(!self_initialized); - - self = triton_map_self(); - result = triton_fakess_node_add(self); - if (result != TRITON_SUCCESS) return result; - - self_initialized = true; - return TRITON_SUCCESS; -} - __blocking void triton_fakess_state_install(void) { triton_debug(debug_system, "triton_fakess_state_install\n"); - triton_state.init_self = triton_fakess_init_self; triton_state.list = triton_fakess_list; triton_state.size = triton_fakess_size; triton_state.load = triton_fakess_load_map; @@ -222,6 +206,7 @@ __blocking void triton_fakess_state_install(void) triton_state.propagation = triton_fakess_propagation; triton_state.shutdown = triton_fakess_shutdown; triton_state.finalize = triton_fakess_finalize; + triton_fakess_self = triton_map_self(); } static struct triton_state_table *lookup_node(triton_node_t node); @@ -351,9 +336,7 @@ triton_ret_t triton_fakess_load_map(void) triton_ret_t error; assert(initialized); - triton_map_get(&list); - triton_list_for_each_entry(map, scratch, list, triton_map_t, link) { @@ -460,18 +443,17 @@ __blocking void triton_fakess_broadcast(triton_msg_tag_t tag, triton_state_peer_t *peer; triton_state_peer_t *scratch; triton_addr_t addr; - triton_node_t self; + triton_string_t s1; + triton_string_t s2; + int i = 0; - triton_string_t tmp2; assert(length < TRITON_FAKESS_SIZE_MESSAGE); - self = triton_map_self(); - triton_list_for_each_entry(peer, scratch, &triton_fakess_peers, triton_state_peer_t, link) { if (triton_cmp_node(peer->node, triton_node_null) && - triton_cmp_node(peer->node, self)) + triton_cmp_node(peer->node, triton_fakess_self)) { triton_map_lookup_node(peer->node, &addr); triton_debug(debug_trace, "send\n"); diff --git a/code/src/fakess/fakess.hae b/code/src/fakess/fakess.hae index 9a4eecf..953820d 100644 --- a/code/src/fakess/fakess.hae +++ b/code/src/fakess/fakess.hae @@ -23,8 +23,6 @@ #define TRITON_FAKESS_UNSET ((void *)1) -__blocking triton_ret_t triton_fakess_init_self(void); - __blocking void triton_fakess_state_install(void); triton_ret_t triton_fakess_list(triton_node_t *list); diff --git a/code/src/mapping/mapping.ae b/code/src/mapping/mapping.ae index 4f069dd..11beba5 100644 --- a/code/src/mapping/mapping.ae +++ b/code/src/mapping/mapping.ae @@ -114,7 +114,8 @@ triton_ret_t triton_map_file(const char *filename) } /** - Initialize the triton_place_map + Initialize the triton_place_map from file + File should contain node numbers in hexadecimal */ triton_ret_t triton_map_read(FILE *file) { @@ -287,6 +288,7 @@ __attribute__((constructor)) void triton_mapping_init_register(void) * Generates a temporary file containing a mapping for servers from <default method>://0 * to <default method>://<servercount-1>. The name of the file is returned by the * function and must be freed by the caller. Returns NULL on failure. + * Produces node numbers in hexadecimal */ char* triton_map_generate_mapfile(int servercount) { @@ -317,7 +319,7 @@ char* triton_map_generate_mapfile(int servercount) for(i=0; i<servercount; i++) { - dprintf(fd, "%s://%llu\t%llu.1\n", method, llu(i), llu((INT64_MAX/big_servercount) * i)); + dprintf(fd, "%s://%llu\t%llX.1\n", method, llu(i), llu((INT64_MAX/big_servercount) * i)); } return(filename); diff --git a/code/src/rebuild/rebuild.ae b/code/src/rebuild/rebuild.ae index 5c7c0c6..58b2f2a 100644 --- a/code/src/rebuild/rebuild.ae +++ b/code/src/rebuild/rebuild.ae @@ -54,6 +54,7 @@ __blocking void enumerate_stub_wus(triton_node_t self, triton_rebuild_wu_stub_t *wu; triton_ret_t ret; triton_node_t other; + char s[TRITON_UINT128_STRLEN]; printf("rebuild: enumerate work units...\n"); @@ -66,6 +67,8 @@ __blocking void enumerate_stub_wus(triton_node_t self, break; for (i = 0; i < actual; i++) { + triton_uint128_to_string(s, TRITON_UINT128_STRLEN, working[i]); + printf("oid: %s\n"); ret = triton_place_lookup(working[i], 3, closest); triton_error_assert(ret); if (contains(closest, 3, failed) && diff --git a/code/src/replicated-osd/tests/module.mk.in b/code/src/replicated-osd/tests/module.mk.in index bf03b77..e5aa6d1 100644 --- a/code/src/replicated-osd/tests/module.mk.in +++ b/code/src/replicated-osd/tests/module.mk.in @@ -2,21 +2,12 @@ DIR := src/replicated-osd/tests ifneq (,$(BUILD_MPI)) -src/replicated-osd/tests/rosd-test-main.o: $(srcdir)/src/replicated-osd/tests/rosd-test-main.ae - $(Q) " AECC $@" - $(E)$(builddir)/maint/aecc \ - -o src/replicated-osd/tests/rosd-test-main.o \ - $< \ - -c $(call MCC_AE,$*) $(VA) $(DA) $(AEPRETTY) \ - -- $(LIBCFLAGS) $(CFLAGS) $(CWARNS) - AETESTSRC += $(DIR)/rosd-create-bench.ae MODCFLAGS_$(DIR)/rosd-create-bench = $(MPICFLAGS) MODLDFLAGS_$(DIR)/rosd-create-bench = $(MPILDFLAGS) MODLIBS_$(DIR)/rosd-create-bench = $(MPILIBS) MODCC_$(DIR)/rosd-create-bench = $(MPICC) -AETESTSRC += $(DIR)/rosd-test-builder.ae MODCFLAGS_$(DIR)/rosd-test-builder = $(MPICFLAGS) MODLDFLAGS_$(DIR)/rosd-test-builder = $(MPILDFLAGS) MODLIBS_$(DIR)/rosd-test-builder = $(MPILIBS) diff --git a/code/src/state/state.ae b/code/src/state/state.ae index ec3e396..b26c8de 100644 --- a/code/src/state/state.ae +++ b/code/src/state/state.ae @@ -72,6 +72,7 @@ __blocking void triton_state_monitor() triton_status_t *status; triton_ret_t ret; bool running; + triton_string_t s; printf("state: monitoring state changes...\n"); @@ -87,7 +88,7 @@ __blocking void triton_state_monitor() triton_error_assert(ret); /* wait for all nodes to post status=NOMINAL */ - triton_timer(1000); + triton_timer(2000); pwait { @@ -105,7 +106,13 @@ __blocking void triton_state_monitor() { peer = entry->node; data = triton_state.lookup(peer, &key); - assert(data); + if (data == NULL) + { + triton_node_to_string(peer, &s); + printf("data not found: node: %s:%s", + triton_string_get(&s), triton_string_get(&key)); + exit(1); + } while (true) { diff --git a/code/src/state/state.hae b/code/src/state/state.hae index d62637a..b812e4b 100644 --- a/code/src/state/state.hae +++ b/code/src/state/state.hae @@ -43,8 +43,6 @@ struct triton_state_funcs { triton_ret_t (*init)(void); - __blocking triton_ret_t (*init_self)(void); - triton_ret_t (*list)(triton_node_t *list); int (*size)(void); diff --git a/code/src/state/tests/module.mk.in b/code/src/state/tests/module.mk.in index 5175b4b..2a2e880 100644 --- a/code/src/state/tests/module.mk.in +++ b/code/src/state/tests/module.mk.in @@ -15,15 +15,8 @@ STATE_TEST_SRC += $(DIR)/rpc-size.ae STATE_TEST_SRC += $(DIR)/rpc-global.ae STATE_TEST_SRC += $(DIR)/rpc-list.ae STATE_TEST_SRC += $(DIR)/rosd-start.ae -# STATE_TEST_SRC += $(DIR)/rosd-global.ae -# STATE_TEST_SRC += $(DIR)/rosd-status.ae -# STATE_TEST_SRC += $(DIR)/rosd-trigger.ae -# STATE_TEST_SRC += $(DIR)/rosd-peers.ae -# STATE_TEST_SRC += $(DIR)/rosd-maint.ae STATE_TEST_SRC += $(DIR)/rosd-injection.ae -# Still needs some work: -# STATE_TEST_SRC += $(DIR)/rosd-recover.ae -# AETESTSRC += $(DIR)/state-client.ae +STATE_TEST_SRC += $(DIR)/rosd-enqueue.ae STATE_TESTS = $(patsubst %.ae,%,$(STATE_TEST_SRC)) @@ -103,4 +96,12 @@ MODCFLAGS_$(DIR)/rosd-injection = $(MPICFLAGS) \ MODLDFLAGS_$(DIR)/rosd-injection = $(MPILDFLAGS) MODLIBS_$(DIR)/rosd-injection = $(MPILIBS) MODCC_$(DIR)/rosd-injection = $(MPICC) -MODLINKWITH_$(DIR)/rosd-injection = src/replicated-osd/tests/rosd-test-builder.o \ No newline at end of file +MODLINKWITH_$(DIR)/rosd-injection = src/replicated-osd/tests/rosd-test-builder.o + +MODCFLAGS_$(DIR)/rosd-enqueue = $(MPICFLAGS) \ + -D ROSD_DISABLE_FAKESS=$(ROSD_DISABLE_FAKESS) \ + -Wunused-variable +MODLDFLAGS_$(DIR)/rosd-enqueue = $(MPILDFLAGS) +MODLIBS_$(DIR)/rosd-enqueue = $(MPILIBS) +MODCC_$(DIR)/rosd-enqueue = $(MPICC) +MODLINKWITH_$(DIR)/rosd-enqueue = src/replicated-osd/tests/rosd-test-builder.o diff --git a/code/src/state/tests/rosd-injection.ae b/code/src/state/tests/rosd-enqueue.ae similarity index 67% copy from code/src/state/tests/rosd-injection.ae copy to code/src/state/tests/rosd-enqueue.ae index 301769b..56fd595 100644 --- a/code/src/state/tests/rosd-injection.ae +++ b/code/src/state/tests/rosd-enqueue.ae @@ -20,6 +20,7 @@ #include "src/net/triton-message-method.hae" #include "src/net/mock/mock-method.h" #include "src/common/triton-init.h" +#include "src/common/traffic-cop.hae" #include "src/common/resources/scheduling/sched.hae" #include "src/common/resources/timer/timer.hae" #include "src/zeroconf/zeroconf.h" @@ -42,6 +43,8 @@ #include "src/replicated-osd/rosd_stubs.hae" #include "src/remote/core_stubs.hae" +// #include "src/replicated-osd/rosd-test-harness.hae" + #include "mpi.h" __blocking triton_ret_t rosd_test_client_fn(int nservers, @@ -56,6 +59,10 @@ __blocking triton_ret_t rosd_test_client_fn(int nservers, uint128_t oid; triton_string_t s; triton_status_t *status; + struct rosd_create_req req; + uint32_t flags = 0; + uint32_t rep_factor = 1; + uint64_t niid; triton_uint128_setzero(oid); @@ -65,6 +72,11 @@ __blocking triton_ret_t rosd_test_client_fn(int nservers, MPI_Comm_rank(client_comm, &rank); printf("rosd-injection: rosd_client_fn(): rank: %i\n", rank); + if (rank != 0) + { + return TRITON_SUCCESS; + } + ret = triton_place_lookup(oid, 1, &svr_node); triton_error_assert(ret); @@ -73,19 +85,39 @@ __blocking triton_ret_t rosd_test_client_fn(int nservers, triton_addr_to_string(svr_addr, &s); printf("addr: %s\n", triton_string_get(&s)); - triton_string_destroy(&s); - // triton_timer(rank*1000-2000); + niid = traffic_cop_rpc_register(); - /* only one client sets the status */ - if (rank == 0) + ret = aer_init_struct_rosd_create_req(&req, &oid, &flags, &rep_factor, &niid); + triton_error_assert(ret); + + aer_destroy_struct_rosd_create_req(&req); + + ret = triton_map_lookup_node(svr_node, &svr_addr); + triton_error_assert(ret); + + ret = remote_rosd_create(AER_DEFAULT_CTX, svr_addr, &req, NULL); + if(ret != TRITON_SUCCESS) { - status = malloc(sizeof(triton_status_t)); - *status = TRITON_STATUS_INJECTED; - ret = remote_state_client_admin_status_set - (AER_DEFAULT_CTX, svr_addr, status, NULL); - triton_error_assert(ret); + printf("error name: %s\n", ret->name); + triton_error_print(ret, "Error: rosd_create() failure."); } + + traffic_cop_rpc_unregister(niid); + + // triton_string_destroy(&s); + + // triton_timer(rank*1000-2000); + + /* only one client sets the status */ + /* if (rank == 0) */ + /* { */ + /* status = malloc(sizeof(triton_status_t)); */ + /* *status = TRITON_STATUS_INJECTED; */ + /* ret = remote_state_client_admin_status_set */ + /* (AER_DEFAULT_CTX, svr_addr, status, NULL); */ + /* triton_error_assert(ret); */ + /* } */ return TRITON_SUCCESS; } diff --git a/code/src/state/tests/rosd-injection.sh b/code/src/state/tests/rosd-enqueue.sh similarity index 67% copy from code/src/state/tests/rosd-injection.sh copy to code/src/state/tests/rosd-enqueue.sh index 7db691b..305665f 100755 --- a/code/src/state/tests/rosd-injection.sh +++ b/code/src/state/tests/rosd-enqueue.sh @@ -11,14 +11,12 @@ OUT=${BIN}.out # export aesop_remote_default_net=fault export triton_traffic_cop_static_timeout=10 -# TODO: skipping this test for now; comment out the following 2 lines to -# re-enable it -# echo "SKIPPING: Test broken as of 2011-12-07" -# exit 0 +PROCS=6 +SERVERS=3 # Test with sorted input... -mpiexec -l -n 4 ${VALGRIND} ${BIN} /tmp/rosd.db /tmp/rosd-log \ - 1 1 2 1 > ${OUT} 2>&1 +mpiexec -l -n ${PROCS} ${VALGRIND} ${BIN} /tmp/rosd.db /tmp/rosd-log \ + 1 1 ${SERVERS} 1 > ${OUT} 2>&1 # [[ $? == 0 ]] || exit 1 diff --git a/code/src/state/tests/rosd-injection.sh b/code/src/state/tests/rosd-injection.sh index 7db691b..e852fd5 100755 --- a/code/src/state/tests/rosd-injection.sh +++ b/code/src/state/tests/rosd-injection.sh @@ -16,6 +16,8 @@ export triton_traffic_cop_static_timeout=10 # echo "SKIPPING: Test broken as of 2011-12-07" # exit 0 +rm -r /tmp/rosd.db /tmp/rosd-log + # Test with sorted input... mpiexec -l -n 4 ${VALGRIND} ${BIN} /tmp/rosd.db /tmp/rosd-log \ 1 1 2 1 > ${OUT} 2>&1 hooks/post-receive -- Triton Repository

1 0

Triton-private Repository branch, master, updated. 3ba288ec312b1621cf063753bdad8b844c2bad52
by noreply＠mcs.anl.gov 13 Mar '12

13 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 3ba288ec312b1621cf063753bdad8b844c2bad52 (commit) from a313cd601816936446489bd52892897b9d4d3f32 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3ba288ec312b1621cf063753bdad8b844c2bad52 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Mar 13 10:14:56 2012 -0500 Use 'asynchronous' ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/stack.pdf | Bin 84227 -> 465675 bytes papers/2011/aesop/stack.svg | 28 ++++++++++++++-------------- 2 files changed, 14 insertions(+), 14 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/stack.pdf b/papers/2011/aesop/stack.pdf index d0b4e2f..c0fbbf6 100644 Binary files a/papers/2011/aesop/stack.pdf and b/papers/2011/aesop/stack.pdf differ diff --git a/papers/2011/aesop/stack.svg b/papers/2011/aesop/stack.svg index 3a53173..a5501e3 100644 --- a/papers/2011/aesop/stack.svg +++ b/papers/2011/aesop/stack.svg @@ -559,7 +559,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-622.72166,-3.1107555)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-636.8949,11.062453)" gradientUnits="userSpaceOnUse" id="linearGradient3293-9" xlink:href="#linearGradient5177-5-7-0-0-3" @@ -1483,7 +1483,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-821.14686,67.755384)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-948.70592,96.101825)" gradientUnits="userSpaceOnUse" id="linearGradient3385" xlink:href="#linearGradient5177-5-7-0-0-3-2" @@ -1798,7 +1798,7 @@ <dc:format>image/svg+xml</dc:format> <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> - <dc:title /> + <dc:title></dc:title> </cc:Work> </rdf:RDF> </metadata> @@ -2103,35 +2103,35 @@ id="tspan4295-5-3" /></text> <path style="fill:url(#linearGradient3171);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="M 992.14568,714.68503 1063.0118,700.5118" + d="m 992.14568,714.68503 99.21262,-14.17324" id="path4299-7-2-0" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3211-5);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 992.14568,601.29921 99.21252,56.69291" + d="m 992.14568,601.29921 99.21262,70.86612" id="path4299-7-2-0-0" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 992.14568,657.99212 70.86612,14.17323" + d="m 992.14568,657.99212 99.21262,28.34644" id="path4299-7-2-0-0-8" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-9);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 382.69686,686.33858 85.03937,70.86614" + d="m 382.69685,686.33856 85.03937,70.86614" id="path4299-7-2-0-0-80" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-2);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 382.69686,686.33858 85.03936,-14.17325" + d="m 382.69686,686.33858 85.03937,-28.34646" id="path4299-7-2-0-0-1" inkscape:connector-type="polyline" inkscape:connector-curvature="0" @@ -2235,7 +2235,7 @@ style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono" id="tspan4379"><tspan style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" - id="tspan4847"> 6</tspan> pbranch { </tspan><tspan + id="tspan4847"> 6</tspan> pbranch { </tspan><tspan sodipodi:role="line" x="16.064432" y="284.24875" @@ -2244,7 +2244,7 @@ id="tspan4859" style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch Bold"> 7</tspan><tspan style="font-weight:bold" - id="tspan4002"> disk_operation</tspan>();</tspan><tspan + id="tspan4002"> disk_operation</tspan>();</tspan><tspan sodipodi:role="line" x="16.064432" y="332.95673" @@ -2253,7 +2253,7 @@ id="tspan4861" style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch Bold"> 8</tspan><tspan style="font-weight:bold" - id="tspan4385"> disk_operation</tspan>(); </tspan><tspan + id="tspan4385"> disk_operation</tspan>(); </tspan><tspan sodipodi:role="line" x="16.064432" y="381.6647" @@ -2369,7 +2369,7 @@ sodipodi:role="line" x="992.14575" y="190.27556" - id="tspan3987-5-3">Non-blocking operations </tspan><tspan + id="tspan3987-5-3">Asynchronous operations </tspan><tspan sodipodi:role="line" x="992.14575" y="240.27556" @@ -2404,14 +2404,14 @@ id="tspan4295-5-2-1" /></text> <path style="fill:url(#linearGradient3345);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 992.14568,771.37795 99.21252,-56.69292" + d="m 992.14568,771.37795 99.21262,-56.69294" id="path4299-7-2-0-6" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3385);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 382.69686,686.33858 85.03937,28.34645" + d="m 382.69685,686.33856 85.03937,28.34645" id="path4299-7-2-0-0-80-5" inkscape:connector-type="polyline" inkscape:connector-curvature="0" hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. a313cd601816936446489bd52892897b9d4d3f32
by noreply＠mcs.anl.gov 13 Mar '12

13 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via a313cd601816936446489bd52892897b9d4d3f32 (commit) from e5f608f0dc391519a2d8dc41805caa5bb270a1c5 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit a313cd601816936446489bd52892897b9d4d3f32 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Mar 13 00:38:29 2012 -0500 Minor edits to aesop document; also added more notes on productivity secion ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/context.tex | 11 +++-- papers/2011/aesop/evaluation.tex | 104 ++++++++++++++++++++++--------------- papers/2011/aesop/intro.tex | 8 ++-- papers/2011/aesop/model.tex | 11 ++-- papers/2011/aesop/related.tex | 2 +- 5 files changed, 80 insertions(+), 56 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/context.tex b/papers/2011/aesop/context.tex index 819ac16..ca8e350 100644 --- a/papers/2011/aesop/context.tex +++ b/papers/2011/aesop/context.tex @@ -62,12 +62,14 @@ such as network transmissions or disk drive operations. A large system consisting of $O(100)$ servers could service $O(1000)$ clients, each performing multiple simultaneous requests. Each request typically requires the server to perform $O(10)$ asynchronous disk or network -operations. +operations. {\color{red}KH: how do we backup that each request typically takes +O(10) operations?} %% \todo{This might be a good place for a mini plot or table.} \begin{table}[h] \caption{Representative scale of current file systems} + \label{tab:counts} \begin{tabular}{|l|l|r|r|} \hline \textbf{Installation} & \textbf{File system} & @@ -95,8 +97,8 @@ triggering the top receive. The server then simultaneously issues four concurrent operations: 1) a send to peer server to forward data to a replica, \todo{removed non-blocking here to avoid term confusion} 2) a receive to accept -the response from the replica server, 3) a timer call to time -the whole operation and perform a timeout if necessary, and 4) a +the response from the replica server, 3) a timer call to +detect a timeout if necessary, and 4) a sequence of metadata operations. Finally, when all four operations complete, the result is posted to the client. @@ -144,7 +146,7 @@ challenging due to the highly asynchronous nature of \emph{systems of operations, caching, and pipelining. These operations require complex concurrency and synchronization mechanisms. Another challenging use case is fault detection, in which an operation posted -to an underlying exceeds a time limit, resulting in a branch to a +to an underlying software layer exceeds a time limit, resulting in a branch to a fault response code path. %% In the server @@ -336,6 +338,7 @@ Additionally, PVFS SMs perform all non-asynchronous computation on a single core, which is an undesirable model given the advent of many-core servers. Finally, PVFS SM simply does not \emph{look like} traditional C code, making it difficult to reason about many development challenges. +{\color{red}KH: this statement sounds a bit odd} These drawbacks introduce a learning curve for researchers and a additional maintenance workload for maintainers. In this work, we seek to diff --git a/papers/2011/aesop/evaluation.tex b/papers/2011/aesop/evaluation.tex index 916648b..06503a9 100644 --- a/papers/2011/aesop/evaluation.tex +++ b/papers/2011/aesop/evaluation.tex @@ -1,11 +1,11 @@ \label{section:evaluation} -In this paper we evaluate Aesop based on three criteria, runtime efficiency, +In this paper we evaluate \aesop based on three criteria, runtime efficiency, memory efficiency and progammer productivity. \subsection{Experiment} -To evaluate the aesop we created a comparison of different server +To evaluate \aesop we created a comparison of different server implementations. We designed a simple server that uses TCP messaging. The server recieves requests from a client, sends or recieves a payload, and then sends an acknowledgement. @@ -20,7 +20,7 @@ generate a highly concurrent workload. The client will execute in a loop generating a specified number of operations to the server. The general flow is that each client process sends a request to the server that contains an optional payload. The client then -waits for the server to send an acknowledgment that also contains an +waits for the server to send an acknowledgment which also contains an optional payload. Each client records the total amount of time taken to execute its workload @@ -77,7 +77,7 @@ The write-null test is identical to the write request, except that the server discards incoming data rather than performing any file I/O. The server will still send an acknowledgement to the client. -The write test had clients each issue 4096 sending 4 KiB of data to be +The write test had clients each issue 4096 requests sending 4 KiB of data to be discarded by the server. \subsubsection{Experiment - Server Types} @@ -85,7 +85,7 @@ discarded by the server. We implemented the same request protocol in multiple server daemons in order to contrast different approaches to concurrent request processing. Each server uses the same fundamental coding style to -the degree possible. One server is implemented using the Aesop language, +the degree possible. One server is implemented using the \aesop language, while all other servers are implemented in C. The pthread library was used in all cases that required explicit threading, while the libev~\cite{libev-web} library was used in all cases that required an @@ -93,12 +93,12 @@ explicit event loop. Each server is setup to report the VmHWM stat when the application exits. -\paragraph*{Aesop} +\paragraph*{\aesop} -The Aesop server is implemented in the Aesop programming language. The server +The \aesop server is implemented in the \aesop programming language. The server uses a {\em lonely pbranch} to service each client. All operations for a client are handled within a single pbranch. The socket and file operations are -performed with blocking Aesop functions that are provided by the Aesop +performed with blocking \aesop functions that are provided by the \aesop standard library. The underlying socket resource uses non-blocking sockets with a thread pool of 12 threads. The file resource uses synchronous IO and a thread pool with 4 threads. @@ -155,7 +155,7 @@ the Argonne Laboratory Computing Resource Center (LCRC). Fusion is a IBM iDataPlex dx360 M2 system. It features 320 compute nodes which each consist of two Intel Nehalem 2.6 GHz Xeon processors and 36 GB of RAM. The compute nodes have hyper threading disabled. The cluster has -an InfiniBand QDR interconnect. Each compute node also a single SATA 7200 RPM +an InfiniBand QDR interconnect. Each compute node also has a single SATA 7200 RPM hard disk for local scratch storage. All disk IO was performed using the local scratch storage and client/server communication was done over the IB network using IPoIB. @@ -168,34 +168,34 @@ used for the clients. We scaled the clients from 16 (1 node) to 1024 (64 nodes). Figure~\ref{fig:write} shows the overall run time of the concurrent write workload for each server implementation as the number of client processes -is scaled from 16 to 1024. In this graph we see that {aesop} does not -perform as well as the other servers for small workloads (taking 3.2 -seconds at the smallest scale, verses 1.9 seconds for the thread-per-op -server). However, {aesop} is the fastest server at the largest scale -(taking 119.8 seconds verses 130.8 seconds for the nearest competitors +is scaled from 16 to 1024. In this graph we see that \aesop does not +perform as well as the other servers for small workloads (delivering 79 +ops/second at the smallest scale, verses 132 ops/seconds for the thread-per-op +server). However, \aesop is the fastest server at the largest scale +(delivering 136 ops/second verses 125 ops/second for the nearest competitors in thread-per-client and thread-per-client-nb). Figure~\ref{fig:read} shows the results of the read experiment. -Aesop performs more favorably at small scale for this workload than in the -previously shown write workload. At the largest scale, Aesop completes the +\aesop performs more favorably at small scale for this workload than in the +previously shown write workload. At the largest scale, \aesop completes the test with 339 ops/sec verses 354 ops/sec for the fastest server (thread-pool), -which is a 4.5% difference. +which is a 4.5\% difference. The event server performs particularly poorly in all cases, ultimately running the largest scale test with only 212 ops/sec. -The small scale results for Aesop may indicate that additional tuning +The small scale results for \aesop may indicate that additional tuning is needed to improve latency for small test runs. The issue is likely -isolated to the write path of the file I/O resource in the Aesop standard -library, as we see asymmetric results in the read and write tests for Aesop +isolated to the write path of the file I/O resource in the \aesop standard +library, as we see asymmetric results in the read and write tests for \aesop in terms of its relative performance. -Aesop is competitive with (and in most cases better than) the other +\aesop is competitive with (and in most cases better than) the other implementations except for the thread-per-client server in both the write-null (Figure~\ref{fig:writenull}) and read-null (Figure~\ref{fig:readnull}) evaluation. We were unable to isolate a concrete reason for this discrepancy in profiling. One notable difference in the two implementations, however, is that the -thread-per-client server uses blocking socket operations, while the Aesop +thread-per-client server uses blocking socket operations, while the \aesop socket resource uses non-blocking operations. Based on this observation, we implemented the thread-per-client-nb server to isolate the impact of non-blocking socket @@ -204,14 +204,14 @@ identical to the thread-per-client implementation except that each socket uses non-blocking operations and polling to transmit and receive data. As seen in these tests, the use of non-blocking operations slows down the thread-per-client server to the point that it is practically equivalent to -the Aesop server at scale. +the \aesop server at scale. -Another notable observation in these graphs is that the Aesop server is +Another notable observation in these graphs is that the \aesop server is competitive at small scale, and in fact is the fastest implementation in the 16 client process read-null test and nearly the fastest in the 16 client process write-null test. This supports the observation from the previous -section that poor Aesop performance at small scale is likely a tuning flaw -in the file resource used in the Aesop standard library, rather than a +section that poor \aesop performance at small scale is likely a tuning flaw +in the file resource used in the \aesop standard library, rather than a fundamental programming language problem. \begin{figure*}[ht] @@ -243,9 +243,11 @@ the minimum and maximum latency, the first quartile latency and third quartile latency. The box represents the first and third quartiles and the whiskers are the minimum and maximum values. The following results are for the 1024 client size selected from the same iteration as the maximum runtime -graphs. Figure~\ref{fig:writelat} shows Aesop offers vary comparative -latency performance as the other configurations and only notably -thread-per-op and event are significantly worse. The relative latency metrics +graphs. Figure~\ref{fig:writelat} shows (in log scale) \aesop offers very +comparative latency performance with respect to the other configurations and +only notably +thread-per-op and event servers are significantly worse. The relative latency +metrics are similar for the other test cases at 1024 clients. \begin{figure}[ht] @@ -257,7 +259,7 @@ are similar for the other test cases at 1024 clients. \subsection{Memory Efficiency} Another aspect of the overall performance is the memory efficiency of each -server implementation. In this section we compare Aesop to the other server +server implementation. In this section we compare \aesop to the other server implementations using the same runtime performance experitment. The memory utilization of each server implementation was captured during the runtime performance experiments. We recorded the VmHWM stat from the @@ -265,11 +267,11 @@ server when the client test was completed. The VmHWM stat is a Linux-specific metric that represents the peak resident set size (RSS) of an executable, where RSS corresponds to the amount of paged-in memory used by the executable. -In Figure~\ref{fig:writemem} we see that thread-pool limits the +In Figure~\ref{fig:writemem} (shown in log scale) we see that thread-pool limits the memory usage as the client work load increases because the thread-pool by design limits the number of requests that can be in progress at once. The other server implementations scale as the number of clients increase. -Although the Aesop server cannot match the thread-pool server in terms of +Although the \aesop server cannot match the thread-pool server in terms of memory usage, it does compare favorably to the thread-per-client and thread-per-op servers. Note that the thread-per-client and thread-per-op models consume virtual memory at a much larger rate due to the number of @@ -286,6 +288,12 @@ in the write test case. \subsection{Productivity} +In order to evaluate our productivity claims, we first look at a quantitative +example. The completixy of each of the server implementations is compared +using standard measures. We then look at producitivty from a qualitiative +view considering the adaptation of a simple code example and anecdotal +evidence while running these tests. + \begin{table} \small \begin{center} @@ -324,12 +332,12 @@ request and acknowledgement structs) as well as helper functions to loop over send and recv were not counted in any of the implementations, as these were similar in all four. -Aesop and thread are very similar in terms of complexity, with the slight +\aesop and thread are very similar in terms of complexity, with the slight increase in the thread model due to function calls needed to create and join threads. Thread pool and event model are both much more complex than the thread or -aesop model. An additional complexity of the event model which is not +\aesop model. An additional complexity of the event model which is not captured by these metrics is the fact that control flow is not preserved across the processing of a given request. For example, servicing a write operation requires 5 disconnected event handlers. So although the event @@ -356,15 +364,17 @@ handle multiple connections concurrently. The example discussed in this section is very simple and differs from real world code by the absence of error handling, timeout handling and request -throttling. As an example, a timeout can be easily implemented by adding a -\code{pbranch} starting a timer which cancels the original \code{pbranch} once -the timeout is reached. The original \code{handleConnection pbranch} will -also cancel the timer when it completes. Listing~\ref{lst:aesop} shows a -simple server design written in aesop which uses a \code{lonely pbranch} to +throttling. As an example, in \aesop a timeout can be easily implemented by adding a +\code{pbranch} which starts a timer which then cancels the original \code{pbranch} once +the timeout is reached. The original \code{handleConnection} pbranch upon +completion will cancel the timer pbranch so that the \code{pwait} block can +proceed. Listing~\ref{lst:aesop} shows a +simple server design written in \aesop which uses a \code{lonely pbranch} to allow accepting new connections while the previous connection is being processed. The server processes a single request from the client and disconnects. Listing~\ref{lst:timeout} demonstrates how to add timeout -handling to the client processing. The same effect is not easily achieved +handling and cancellation to the client processing. +The same effect is not easily achieved using the event or threaded models, since neither model offers any help in cancelling an outstanding call. @@ -415,6 +425,16 @@ __blocking void handleConnectionWithTO(int s, int timeout) \end{minipage} \end{figure*} -\color{black} -% +Another example we indirectly evaluted during our runtime analysis was the +flexibilty of changing the underlying concurrency models without modifing +the \aesop server source. In our initial testing, we wanted to tune \aesop +to perform well on a few test systems to estabilsh reasonable default settings. +During this testing we tried different strategies for the thread-pool +implementation, we experimented with multiple threads driving the event loop +and we using different file I/O APIs. All of these changes were done within +the \aesop runtime library and never required changing the \aesop server +test code. This type of flexibility allows experimentation to determine the +best tuning strategy for any given system without redesigning the core +software. +\color{black} diff --git a/papers/2011/aesop/intro.tex b/papers/2011/aesop/intro.tex index 9942f89..8e7df47 100644 --- a/papers/2011/aesop/intro.tex +++ b/papers/2011/aesop/intro.tex @@ -5,7 +5,7 @@ software to provide shared services for applications. Examples include file storage, scheduling, security, checkpointing, web services, membership management, and fault detection. Today's systems demand that these services scale to hundreds of thousands of concurrent clients, -with millions of clients expected in the near future. Developing system +with millions of clients expected in the near future. {\color{red}KH: need citation. Reference to Table~\ref{tab:counts}} Developing system software for this environment is a complex task due to a number of competing software engineering requirements: @@ -57,7 +57,7 @@ to make optimal use of local resources. For example, even if there are 1,000 client requests in flight at once, the server may have only a few processing cores and a storage device that can only effectively handle a few operations at a time. This disconnect can cause a seemingly simple thread -model to evolve into a more complex framework in order to local resource +model to evolve into a more complex framework in order to take local resource scheduling constraints into account. Event-driven models can be used to address some of the common @@ -130,8 +130,8 @@ used in the case study The remainder of this paper is organized as follows. Sections~\ref{section:context} and~\ref{section:related} describe the challenges of distributed system software development and related work in -addressing those challenges. Sections~\ref{section:model} -and~\ref{section:language} present the \aesop programming model and +addressing those challenges. Section~\ref{section:model} +presents the \aesop programming model and programming language. Section~\ref{section:evaluation} describes a simple example distributed service and uses it as a case study to contrast \aesop against other models in terms of performance, memory efficiency, and diff --git a/papers/2011/aesop/model.tex b/papers/2011/aesop/model.tex index e477df5..62f02ff 100644 --- a/papers/2011/aesop/model.tex +++ b/papers/2011/aesop/model.tex @@ -7,8 +7,8 @@ program model. Concurrency is expressed in \aesop through the use of {\em parallel branches} or {\em pbranches}. A parallel branch groups a list of statements and -supports branch-scoped variables, much like a regular function. However, when -multiple which will be executed sequentially, as in a normal C program. +supports branch-scoped variables, much like a regular function. {\color{red}However, when +multiple which will be executed sequentially, as in a normal C program.} However, when multiple pbranches are active, the \aesop language only enforces sequential execution on the scope of a single pbranch. Statements from other pbranches might interleave execution or might execute concurrently (using a @@ -56,7 +56,7 @@ lonely pbranches cannot be cancelled or synchronized with. \subsection{Blocking Functions} -\aesop adds extends the C programming language with a new function type: {\em +\aesop extends the C programming language with a new function type: {\em blocking} functions. The blocking qualifyier is part of the type. A pointer to a regular C function is not compatible with a function pointer to a blocking function with the same function arguments. Blocking functions differ from @@ -65,7 +65,7 @@ In addition, blocking functions support {\em cancellation}. Functions can be made blocking by adding the \code{__blocking} qualifyer to the function declaration. However, blocking functions can only be -called (either direct or indirect through a blocking function pointer) by +called (either directly or indirectly through a blocking function pointer) by other blocking functions. This effectively requires the starting function of an \aesop program to be blocking. \color{red}PHC: what about bindings like we use for the key/val API? This is perhaps important to show that Aesop @@ -127,7 +127,7 @@ happening within the aesop runtime engine.\color{black} } \end{figure} -Listing~\ref{lst:parallelfor} shows a common aesop code pattern. +Figure~\ref{fig:example} shows a common aesop code pattern. Network servers, having a finite set of resources, have to protect against unbounded resource consumption by misbehaving or failing peers. This is typically done by placing an upper limit on the time resources are dedicated @@ -165,6 +165,7 @@ exceeded. \begin{figure} \input listings/listing-1.ae.tex \caption{Aesop example code.} +\label{fig:example} \end{figure} \color{red} diff --git a/papers/2011/aesop/related.tex b/papers/2011/aesop/related.tex index e4e1b84..5d4cee5 100644 --- a/papers/2011/aesop/related.tex +++ b/papers/2011/aesop/related.tex @@ -21,7 +21,7 @@ maximum use of available CPU resources, while \aesop instead optimizes for maximum use of underlying devices that may or may not require significant CPU activity. The level of \aesop concurrency is not strongly correlated with the number of CPU cores available, though it does utilize multiple -cores (if available) to improve latency. +cores (if available) to improve performance. Grand Central Dispatch (GCD)~\cite{gcd-web} is a popular programming model introduced by Apple Inc. that allows developers to describe hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. e5f608f0dc391519a2d8dc41805caa5bb270a1c5
by noreply＠mcs.anl.gov 12 Mar '12

12 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via e5f608f0dc391519a2d8dc41805caa5bb270a1c5 (commit) from 292f1b3acb6894678e0bd4fa1aed4fcd8d7e5704 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e5f608f0dc391519a2d8dc41805caa5bb270a1c5 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Mon Mar 12 17:37:29 2012 -0500 Add pbranch{}; add FORK/JOIN/OR annotations ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/stack.pdf | Bin 74661 -> 84227 bytes papers/2011/aesop/stack.svg | 818 +++++++++++++++++++++++++++++++++---------- 2 files changed, 631 insertions(+), 187 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/stack.pdf b/papers/2011/aesop/stack.pdf index 2613eda..d0b4e2f 100644 Binary files a/papers/2011/aesop/stack.pdf and b/papers/2011/aesop/stack.pdf differ diff --git a/papers/2011/aesop/stack.svg b/papers/2011/aesop/stack.svg index 24a40e6..3a53173 100644 --- a/papers/2011/aesop/stack.svg +++ b/papers/2011/aesop/stack.svg @@ -11,7 +11,7 @@ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" width="1625" - height="1040" + height="1200" id="svg2" version="1.1" inkscape:version="0.48.2 r9819" @@ -221,7 +221,7 @@ x2="940" y2="527.5" gradientUnits="userSpaceOnUse" - gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-893.5879,272.93639)" /> + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-989.49336,286.07025)" /> <linearGradient inkscape:collect="always" xlink:href="#linearGradient5177-5" @@ -349,7 +349,7 @@ x2="940" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,42.947593,-78.858831)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,163.89252,28.936463)" gradientUnits="userSpaceOnUse" id="linearGradient3171" xlink:href="#linearGradient5177-5-7-0" @@ -431,7 +431,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,113.81373,62.873453)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,234.75866,170.66874)" gradientUnits="userSpaceOnUse" id="linearGradient3211-5" xlink:href="#linearGradient5177-5-7-0-0" @@ -513,7 +513,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,57.12082,-93.032061)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,178.06575,14.763233)" gradientUnits="userSpaceOnUse" id="linearGradient3293" xlink:href="#linearGradient5177-5-7-0-0-0" @@ -559,7 +559,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,57.12082,-93.032061)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-622.72166,-3.1107555)" gradientUnits="userSpaceOnUse" id="linearGradient3293-9" xlink:href="#linearGradient5177-5-7-0-0-3" @@ -605,7 +605,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,57.12082,-93.032061)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-622.72166,-3.1107555)" gradientUnits="userSpaceOnUse" id="linearGradient3293-2" xlink:href="#linearGradient5177-5-7-0-0-9" @@ -1011,7 +1011,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,71.294048,-234.76434)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,170.97913,-229.88244)" gradientUnits="userSpaceOnUse" id="linearGradient3293-8" xlink:href="#linearGradient5177-5-7-0-0-51" @@ -1057,7 +1057,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,71.294048,-234.76434)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,170.97913,-229.88244)" gradientUnits="userSpaceOnUse" id="linearGradient3293-1" xlink:href="#linearGradient5177-5-7-0-0-1" @@ -1103,7 +1103,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-13.745322,-248.93757)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-693.5878,-159.01627)" gradientUnits="userSpaceOnUse" id="linearGradient3293-66" xlink:href="#linearGradient5177-5-7-0-0-94" @@ -1437,7 +1437,7 @@ x2="940" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,42.947594,-50.512371)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,163.89252,57.282923)" gradientUnits="userSpaceOnUse" id="linearGradient3345" xlink:href="#linearGradient5177-5-7-0-3" @@ -1483,11 +1483,273 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-141.30438,-22.165921)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-821.14686,67.755384)" gradientUnits="userSpaceOnUse" id="linearGradient3385" xlink:href="#linearGradient5177-5-7-0-0-3-2" inkscape:collect="always" /> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-97" + style="overflow:visible"> + <path + id="path4855-359" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="marker4462" + style="overflow:visible"> + <path + id="path4464" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="marker4470" + style="overflow:visible"> + <path + id="path4472" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="507.75555" + x2="913.00281" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-84.139,-73.976909)" + gradientUnits="userSpaceOnUse" + id="linearGradient3293-9-4" + xlink:href="#linearGradient5177-5-7-0-0-3-3" + inkscape:collect="always" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-5-7-0-0-3-3"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-4-5-6-6-00-8" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-3-4-7-1-7-1" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="marker4478" + style="overflow:visible"> + <path + id="path4480" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5177-2" + id="linearGradient5183-1" + x1="1040" + y1="647.5" + x2="940" + y2="527.5" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-1003.6666,286.07027)" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-2"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-8" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-1" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-70" + style="overflow:visible"> + <path + id="path4855-86" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="527.5" + x2="940" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-961.14691,328.58993)" + gradientUnits="userSpaceOnUse" + id="linearGradient4568" + xlink:href="#linearGradient5177-2" + inkscape:collect="always" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5177-0" + id="linearGradient5183-4" + x1="1040" + y1="647.5" + x2="940" + y2="527.5" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-1003.6666,286.07027)" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-0"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-5" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-7" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-99" + style="overflow:visible"> + <path + id="path4855-14" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="527.5" + x2="940" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-961.14691,328.58993)" + gradientUnits="userSpaceOnUse" + id="linearGradient4568-2" + xlink:href="#linearGradient5177-0" + inkscape:collect="always" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5177-3" + id="linearGradient5183-7" + x1="1040" + y1="647.5" + x2="940" + y2="527.5" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-1003.6666,286.07027)" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-3"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-6" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-35" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-76" + style="overflow:visible"> + <path + id="path4855-42" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="527.5" + x2="940" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-961.14691,328.58993)" + gradientUnits="userSpaceOnUse" + id="linearGradient4568-4" + xlink:href="#linearGradient5177-3" + inkscape:collect="always" /> + <linearGradient + inkscape:collect="always" + xlink:href="#linearGradient5177-4" + id="linearGradient5183-67" + x1="1040" + y1="647.5" + x2="940" + y2="527.5" + gradientUnits="userSpaceOnUse" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-1003.6666,286.07027)" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-4"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-3" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-0" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-992" + style="overflow:visible"> + <path + id="path4855-39" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="527.5" + x2="940" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-436.73748,311.03088)" + gradientUnits="userSpaceOnUse" + id="linearGradient4568-0" + xlink:href="#linearGradient5177-4" + inkscape:collect="always" /> </defs> <sodipodi:namedview id="base" @@ -1497,13 +1759,13 @@ inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:zoom="0.6883404" - inkscape:cx="816.49415" + inkscape:cx="976.7537" inkscape:cy="515.86812" inkscape:document-units="px" inkscape:current-layer="layer1" showgrid="true" - inkscape:window-width="1600" - inkscape:window-height="876" + inkscape:window-width="1920" + inkscape:window-height="1176" inkscape:window-x="0" inkscape:window-y="24" inkscape:window-maximized="1" @@ -1536,7 +1798,7 @@ <dc:format>image/svg+xml</dc:format> <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> - <dc:title></dc:title> + <dc:title /> </cc:Work> </rdf:RDF> </metadata> @@ -1544,21 +1806,21 @@ inkscape:label="Layer 1" inkscape:groupmode="layer" id="layer1" - transform="translate(42.5,-57.362183)"> + transform="translate(42.5,102.63782)"> <rect style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:3.17159414;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.17159461, 3.17159461;stroke-dashoffset:0" id="rect5533" - width="1601.1024" - height="335.27554" + width="1601.5748" + height="311.81104" x="-28.326771" - y="530.43304" + y="558.77948" rx="43.243244" - ry="14.887338" /> + ry="14.173251" /> <rect style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:2.57646298;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:2.57646335, 2.57646335;stroke-dashoffset:0" id="rect5533-7" - width="1600.4725" - height="141.7323" + width="1597.0078" + height="124.25199" x="-23.759842" y="944.76373" rx="43.243244" @@ -1675,7 +1937,7 @@ </g> <path style="fill:url(#linearGradient5183);fill-opacity:1;stroke:#000000;stroke-width:2.16024685;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 266.00391,673.20468 56.69292,0" + d="m 170.09844,686.33858 28.34645,0" id="path4299" inkscape:connector-type="polyline" inkscape:connector-curvature="0" @@ -1704,17 +1966,17 @@ <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7" - y="687.37793" - x="161.79912" + y="700.51178" + x="80.066887" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="687.37793" - x="166.79131" + y="700.51178" + x="85.059074" sodipodi:role="line" id="tspan4293-3" style="font-weight:bold">{ state:1 } </tspan><tspan - y="732.37793" - x="161.79912" + y="745.51178" + x="80.066887" sodipodi:role="line" id="tspan4295-5" /></text> <flowRoot @@ -1730,167 +1992,167 @@ id="flowPara4024" /></flowRoot> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-0" - y="687.37793" - x="431.09045" + y="700.51178" + x="292.66531" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="687.37793" - x="436.08264" + y="700.51178" + x="297.6575" sodipodi:role="line" id="tspan4293-3-0" style="font-weight:bold">{ state:2 } </tspan><tspan - y="732.37793" - x="431.09045" + y="745.51178" + x="292.66531" sodipodi:role="line" id="tspan4295-5-0" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-5" - y="596.4173" - x="788.25586" + y="615.47247" + x="590.3031" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="596.4173" - x="793.24805" + y="615.47247" + x="595.29529" sodipodi:role="line" id="tspan4293-3-4" style="font-weight:bold">{ state:3[0] } </tspan><tspan - y="641.4173" - x="788.25586" + y="660.47247" + x="590.3031" sodipodi:role="line" id="tspan4295-5-1" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-4" - y="653.11017" - x="788.25586" + y="672.16534" + x="590.3031" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="653.11017" - x="793.24805" + y="672.16534" + x="595.29529" sodipodi:role="line" id="tspan4293-3-1" style="font-weight:bold">{ state:3[1] } </tspan><tspan - y="698.11017" - x="788.25586" + y="717.16534" + x="590.3031" sodipodi:role="line" id="tspan4295-5-7" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-2" - y="709.8031" - x="788.25586" + y="728.85828" + x="590.3031" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="709.8031" - x="793.24805" + y="728.85828" + x="595.29529" sodipodi:role="line" id="tspan4293-3-03" style="font-weight:bold">{ state:3[2] } </tspan><tspan - y="754.8031" - x="788.25586" + y="773.85828" + x="590.3031" sodipodi:role="line" id="tspan4295-5-2" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-49" - y="687.37793" - x="1068.8857" + y="700.51184" + x="1185.5786" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="687.37793" - x="1073.8779" + y="700.51184" + x="1190.5708" sodipodi:role="line" id="tspan4293-3-8" - style="font-weight:bold">{ state:4 } </tspan><tspan - y="732.37793" - x="1068.8857" + style="font-weight:bold">{ state:5 } </tspan><tspan + y="745.51184" + x="1185.5786" sodipodi:role="line" id="tspan4295-5-19" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-3" - y="638.93695" - x="1355.1849" + y="643.81891" + x="1454.87" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="638.93695" - x="1360.1771" + y="643.81891" + x="1459.8622" sodipodi:role="line" id="tspan4293-3-9" - style="font-weight:bold">{ state:5a } </tspan><tspan - y="683.93695" - x="1355.1849" + style="font-weight:bold">{ state:6a } </tspan><tspan + y="688.81891" + x="1454.87" sodipodi:role="line" id="tspan4295-5-5" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-9" - y="738.14954" - x="1355.1849" + y="757.20471" + x="1454.87" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="738.14954" - x="1360.1771" + y="757.20471" + x="1459.8622" sodipodi:role="line" id="tspan4293-3-10" - style="font-weight:bold">{ state:5b } </tspan><tspan - y="783.14954" - x="1355.1849" + style="font-weight:bold">{ state:6b } </tspan><tspan + y="802.20471" + x="1454.87" sodipodi:role="line" id="tspan4295-5-3" /></text> <path style="fill:url(#linearGradient3171);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 899.54722,702.71649 63.77953,-14.17323" + d="M 992.14568,714.68503 1063.0118,700.5118" id="path4299-7-2-0" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3211-5);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 899.54722,596.41728 70.86614,49.60629" + d="m 992.14568,601.29921 99.21252,56.69291" id="path4299-7-2-0-0" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 899.54722,646.02357 63.77953,21.25985" + d="m 992.14568,657.99212 70.86612,14.17323" id="path4299-7-2-0-0-8" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-9);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 523.95667,674.37003 141.73228,21.25984" + d="m 382.69686,686.33858 85.03937,70.86614" id="path4299-7-2-0-0-80" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-2);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="M 523.95667,674.37003 665.68895,653.11019" + d="m 382.69686,686.33858 85.03936,-14.17325" id="path4299-7-2-0-0-1" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-8);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 1162.2244,672.16533 84.567,37.63777" + d="m 1275.6102,686.33858 70.8661,42.51968" id="path4299-7-2-0-0-17" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-1);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 1162.2244,672.16533 84.567,-33.22837" + d="m 1275.6102,686.33858 70.8663,-42.51972" id="path4299-7-2-0-0-4" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-66);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="M 523.95667,674.37003 662.85431,602.33854" + d="m 382.69686,686.33858 85.03937,-70.86615" id="path4299-7-2-0-0-47" inkscape:connector-type="polyline" inkscape:connector-curvature="0" @@ -1898,242 +2160,424 @@ <text xml:space="preserve" style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" - x="790.41345" - y="828.0708" + x="793.72046" + y="856.4173" id="text3983-3" sodipodi:linespacing="125%"><tspan sodipodi:role="line" - x="790.41345" - y="828.0708" + x="793.72046" + y="856.4173" id="tspan3989-7" style="text-align:center;text-anchor:middle">States and transitions</tspan><tspan sodipodi:role="line" - x="790.41345" - y="878.0708" + x="793.72046" + y="906.41724" id="tspan3987-5" /></text> <rect - style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:3.37171936;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.37171988, 3.37171988;stroke-dashoffset:0" + style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:3.69353342;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.69353405, 3.69353405;stroke-dashoffset:0" id="rect5533-1" width="1600" - height="425.19687" - x="-28.326771" - y="76.889725" + height="623.62225" + x="-26.751984" + y="-79.015747" rx="43.243244" - ry="16.825371" /> + ry="20.190445" /> <text xml:space="preserve" style="font-size:40.75279617px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" - x="14.460023" - y="131.11508" + x="16.064432" + y="-7.9990673" id="text3844" sodipodi:linespacing="125%" transform="scale(0.98152775,1.0188199)"><tspan sodipodi:role="line" id="tspan3846" - x="14.460023" - y="131.11508" + x="16.064432" + y="-7.9990673" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + id="tspan4821" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch"> 1</tspan><tspan style="font-weight:bold" - id="tspan3982"><tspan - style="font-weight:normal;-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3412">1</tspan> aesop service_function</tspan>() { </tspan><tspan + id="tspan3982"> aesop service_function</tspan>() { </tspan><tspan sodipodi:role="line" - x="14.460023" - y="176.11508" + x="16.064432" + y="40.708904" id="tspan3848" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan - style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3418">2</tspan> char data[MAX]; </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan3418"> 2</tspan> char data[MAX]; </tspan><tspan sodipodi:role="line" - x="14.460023" - y="221.11508" + x="16.064432" + y="89.41687" id="tspan3850" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + id="tspan4823" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch"> 3</tspan><tspan style="font-weight:bold" - id="tspan4004"><tspan - style="font-weight:normal;-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3414">3</tspan> network_operation</tspan>();</tspan><tspan + id="tspan4004"> network_operation</tspan>();</tspan><tspan sodipodi:role="line" - x="14.460023" - y="266.11508" + x="16.064432" + y="138.12485" id="tspan3852" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan - style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3422">4</tspan> for ( ... ) { </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan3422"> 4</tspan> pwait { </tspan><tspan sodipodi:role="line" - x="14.460023" - y="311.11508" + x="16.064432" + y="186.83281" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono" + id="tspan4834"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan4849"> 5</tspan> for ( ... ) {</tspan><tspan + sodipodi:role="line" + x="16.064432" + y="235.54079" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono" + id="tspan4379"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan4847"> 6</tspan> pbranch { </tspan><tspan + sodipodi:role="line" + x="16.064432" + y="284.24875" id="tspan3854" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + id="tspan4859" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch Bold"> 7</tspan><tspan style="font-weight:bold" - id="tspan4002"><tspan - style="font-weight:normal;-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3410">5</tspan> disk_operation</tspan>();</tspan><tspan + id="tspan4002"> disk_operation</tspan>();</tspan><tspan sodipodi:role="line" - x="14.460023" - y="356.11508" - id="tspan3856" - style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan - style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3426">6</tspan> }</tspan><tspan + x="16.064432" + y="332.95673" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono" + id="tspan4383"><tspan + id="tspan4861" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch Bold"> 8</tspan><tspan + style="font-weight:bold" + id="tspan4385"> disk_operation</tspan>(); </tspan><tspan sodipodi:role="line" - x="14.460023" - y="401.11508" + x="16.064432" + y="381.6647" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono" + id="tspan4830"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan4841"> 9</tspan> }}} </tspan><tspan + sodipodi:role="line" + x="16.064432" + y="430.37265" id="tspan3858" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan - style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3428">7</tspan> if (<tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan4672">10</tspan> if (c) { <tspan style="font-weight:bold" - id="tspan4000">db_operation</tspan>()) { ... } </tspan><tspan + id="tspan4438">db_operation</tspan>(); } </tspan><tspan sodipodi:role="line" - x="14.460023" - y="446.11508" + x="16.064432" + y="479.08063" id="tspan3860" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan - style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" - id="tspan3430">8</tspan> else { ... } </tspan></text> + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-family:Courier 10 Pitch;-inkscape-font-specification:Courier 10 Pitch" + id="tspan4674">11</tspan> else { <tspan + style="font-weight:bold" + id="tspan4440">fail</tspan>(); } </tspan></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-37" - y="176.10233" - x="829.67316" + y="62.716511" + x="831.24799" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="176.10233" - x="834.66534" + y="62.716511" + x="836.24017" sodipodi:role="line" id="tspan4293-3-3" style="font-weight:bold">{ state:1 } </tspan><tspan - y="221.10233" - x="829.67316" + y="107.71651" + x="831.24799" sodipodi:role="line" id="tspan4295-5-4" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-0-6" - y="261.14166" - x="829.67316" + y="161.92911" + x="831.24799" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="261.14166" - x="834.66534" + y="161.92911" + x="836.24017" sodipodi:role="line" id="tspan4293-3-0-4" style="font-weight:bold">{ state:2 } </tspan><tspan - y="306.14166" - x="829.67316" + y="206.92912" + x="831.24799" sodipodi:role="line" id="tspan4295-5-0-4" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-5-3" - y="317.83459" - x="829.67316" + y="303.66141" + x="831.24799" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="317.83459" - x="834.66534" + y="303.66141" + x="836.24017" sodipodi:role="line" id="tspan4293-3-4-7" style="font-weight:bold">{ state:3[...] } </tspan><tspan - y="362.83459" - x="829.67316" + y="348.66141" + x="831.24799" sodipodi:role="line" id="tspan4295-5-1-7" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-5-3-5" - y="374.52753" - x="829.67316" + y="402.87399" + x="831.24799" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="374.52753" - x="834.66534" + y="402.87399" + x="836.24017" sodipodi:role="line" id="tspan4293-3-4-7-7" - style="font-weight:bold">{ state:4 } </tspan><tspan - y="419.52753" - x="829.67316" + style="font-weight:bold">{ state:5 } </tspan><tspan + y="447.87399" + x="831.24799" sodipodi:role="line" id="tspan4295-5-1-7-5" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-3-7" - y="431.22043" - x="829.67316" + y="459.56689" + x="831.24799" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="431.22043" - x="834.66534" + y="459.56689" + x="836.24017" sodipodi:role="line" id="tspan4293-3-9-7" - style="font-weight:bold">{ state:5a,b } </tspan><tspan - y="476.22043" - x="829.67316" + style="font-weight:bold">{ state:6a } </tspan><tspan + y="504.56689" + x="831.24799" sodipodi:role="line" id="tspan4295-5-5-7" /></text> <text xml:space="preserve" style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" - x="990.57092" - y="218.62201" + x="992.14575" + y="190.27556" id="text3983-3-8" sodipodi:linespacing="125%"><tspan sodipodi:role="line" - x="990.57092" - y="218.62201" + x="992.14575" + y="190.27556" id="tspan3987-5-3">Non-blocking operations </tspan><tspan sodipodi:role="line" - x="990.57092" - y="268.62201" + x="992.14575" + y="240.27556" id="tspan4042">(shown in bold) and </tspan><tspan sodipodi:role="line" - x="990.57092" - y="318.62201" + x="992.14575" + y="290.27557" id="tspan4048">service code logic (states)</tspan><tspan sodipodi:role="line" - x="990.57092" - y="368.62201" + x="992.14575" + y="340.27557" id="tspan4044">expressed as sequential </tspan><tspan sodipodi:role="line" - x="990.57092" - y="418.62201" + x="992.14575" + y="390.27557" id="tspan4040">control flow </tspan></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-2-9" - y="766.49597" - x="788.25586" + y="785.55115" + x="590.3031" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="766.49597" - x="793.24805" + y="785.55115" + x="595.29529" sodipodi:role="line" id="tspan4293-3-03-1" style="font-weight:bold">{ state:3[...] } </tspan><tspan - y="811.49597" - x="788.25586" + y="830.55115" + x="590.3031" sodipodi:role="line" id="tspan4295-5-2-1" /></text> <path style="fill:url(#linearGradient3345);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="M 906.63383,752.32279 970.41336,709.8031" + d="m 992.14568,771.37795 99.21252,-56.69292" id="path4299-7-2-0-6" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3385);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 523.95667,674.37003 141.73228,70.86614" + d="m 382.69686,686.33858 85.03937,28.34645" id="path4299-7-2-0-0-80-5" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" - d="m 56.712598,105.2362 c 0,396.85039 0,382.67716 0,382.67716" + d="m 72.460597,-36.496065 c 0,396.850385 0,538.582675 0,538.582675" id="path3432" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> + <text + xml:space="preserve" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="1346.4763" + y="700.51184" + id="text3331" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan3333" + x="1346.4763" + y="700.51184">OR</tspan></text> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-5-7" + y="615.47247" + x="873.76764" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="615.47247" + x="878.75983" + sodipodi:role="line" + id="tspan4293-3-4-5" + style="font-weight:bold">{ state:4[0] } </tspan><tspan + y="660.47247" + x="873.76764" + sodipodi:role="line" + id="tspan4295-5-1-3" /></text> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-4-0" + y="672.16534" + x="873.76764" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="672.16534" + x="878.75983" + sodipodi:role="line" + id="tspan4293-3-1-9" + style="font-weight:bold">{ state:4[1] } </tspan><tspan + y="717.16534" + x="873.76764" + sodipodi:role="line" + id="tspan4295-5-7-2" /></text> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-2-3" + y="728.85828" + x="873.76764" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="728.85828" + x="878.75983" + sodipodi:role="line" + id="tspan4293-3-03-3" + style="font-weight:bold">{ state:4[2] } </tspan><tspan + y="773.85828" + x="873.76764" + sodipodi:role="line" + id="tspan4295-5-2-2" /></text> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-2-9-5" + y="785.55115" + x="873.76764" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="785.55115" + x="878.75983" + sodipodi:role="line" + id="tspan4293-3-03-1-1" + style="font-weight:bold">{ state:4[...] } </tspan><tspan + y="830.55115" + x="873.76764" + sodipodi:role="line" + id="tspan4295-5-2-1-3" /></text> + <path + style="fill:url(#linearGradient4568);fill-opacity:1;stroke:#000000;stroke-width:2.16024685;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" + d="m 708.68111,771.37795 42.51969,0" + id="path4299-2" + inkscape:connector-type="polyline" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:url(#linearGradient4568-2);fill-opacity:1;stroke:#000000;stroke-width:2.16024685;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" + d="m 708.68111,714.68503 42.51969,0" + id="path4299-8" + inkscape:connector-type="polyline" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:url(#linearGradient4568-4);fill-opacity:1;stroke:#000000;stroke-width:2.16024685;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" + d="m 708.68111,657.99212 42.51969,0" + id="path4299-7" + inkscape:connector-type="polyline" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:url(#linearGradient4568-0);fill-opacity:1;stroke:#000000;stroke-width:2.16024685;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" + d="m 708.68111,601.29921 42.51969,-3e-5" + id="path4299-28" + inkscape:connector-type="polyline" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-5-3-8" + y="346.18109" + x="831.24799" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="346.18109" + x="836.24017" + sodipodi:role="line" + id="tspan4293-3-4-7-77" + style="font-weight:bold">{ state:4[...] } </tspan><tspan + y="391.18109" + x="831.24799" + sodipodi:role="line" + id="tspan4295-5-1-7-6" /></text> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-3-7-9" + y="502.08658" + x="831.24799" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="502.08658" + x="836.24017" + sodipodi:role="line" + id="tspan4293-3-9-7-8" + style="font-weight:bold">{ state:6b } </tspan><tspan + y="547.08655" + x="831.24799" + sodipodi:role="line" + id="tspan4295-5-5-7-7" /></text> + <text + xml:space="preserve" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="1048.8386" + y="615.47241" + id="text3331-1" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan3333-0" + x="1048.8386" + y="615.47241">JOIN</tspan></text> + <text + xml:space="preserve" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="340.17715" + y="615.47241" + id="text3331-1-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan3333-0-2" + x="340.17715" + y="615.47241">FORK</tspan></text> </g> </svg> hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. 292f1b3acb6894678e0bd4fa1aed4fcd8d7e5704
by noreply＠mcs.anl.gov 09 Mar '12

09 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 292f1b3acb6894678e0bd4fa1aed4fcd8d7e5704 (commit) via 52c6dd62d476ad8b6be2d979ced8b5ac7215830c (commit) from fb54078b154c8bbedff6ab9f47d235036e8fa789 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 292f1b3acb6894678e0bd4fa1aed4fcd8d7e5704 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Fri Mar 9 12:40:24 2012 -0600 Minor fix commit 52c6dd62d476ad8b6be2d979ced8b5ac7215830c Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Fri Mar 9 12:38:52 2012 -0600 Improvements suggested by Dries ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/stack.pdf | Bin 55756 -> 74661 bytes papers/2011/aesop/stack.svg | 459 ++++++++++++++++++++++++++++--------------- 2 files changed, 299 insertions(+), 160 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/stack.pdf b/papers/2011/aesop/stack.pdf index ba06b02..2613eda 100644 Binary files a/papers/2011/aesop/stack.pdf and b/papers/2011/aesop/stack.pdf differ diff --git a/papers/2011/aesop/stack.svg b/papers/2011/aesop/stack.svg index dbcf391..24a40e6 100644 --- a/papers/2011/aesop/stack.svg +++ b/papers/2011/aesop/stack.svg @@ -11,7 +11,7 @@ xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" width="1625" - height="1160" + height="1040" id="svg2" version="1.1" inkscape:version="0.48.2 r9819" @@ -221,7 +221,7 @@ x2="940" y2="527.5" gradientUnits="userSpaceOnUse" - gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-935.63512,150.25925)" /> + gradientTransform="matrix(0.99477643,-0.11909063,0.10207768,1.1605725,-893.5879,272.93639)" /> <linearGradient inkscape:collect="always" xlink:href="#linearGradient5177-5" @@ -349,7 +349,7 @@ x2="940" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,0.90036935,-201.53597)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,42.947593,-78.858831)" gradientUnits="userSpaceOnUse" id="linearGradient3171" xlink:href="#linearGradient5177-5-7-0" @@ -431,7 +431,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,71.766512,-59.803692)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,113.81373,62.873453)" gradientUnits="userSpaceOnUse" id="linearGradient3211-5" xlink:href="#linearGradient5177-5-7-0-0" @@ -513,7 +513,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,15.073596,-215.7092)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,57.12082,-93.032061)" gradientUnits="userSpaceOnUse" id="linearGradient3293" xlink:href="#linearGradient5177-5-7-0-0-0" @@ -559,7 +559,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,15.073596,-215.7092)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,57.12082,-93.032061)" gradientUnits="userSpaceOnUse" id="linearGradient3293-9" xlink:href="#linearGradient5177-5-7-0-0-3" @@ -605,7 +605,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,15.073596,-215.7092)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,57.12082,-93.032061)" gradientUnits="userSpaceOnUse" id="linearGradient3293-2" xlink:href="#linearGradient5177-5-7-0-0-9" @@ -1011,7 +1011,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,29.246824,-357.44148)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,71.294048,-234.76434)" gradientUnits="userSpaceOnUse" id="linearGradient3293-8" xlink:href="#linearGradient5177-5-7-0-0-51" @@ -1057,7 +1057,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,29.246824,-357.44148)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,71.294048,-234.76434)" gradientUnits="userSpaceOnUse" id="linearGradient3293-1" xlink:href="#linearGradient5177-5-7-0-0-1" @@ -1103,7 +1103,7 @@ x2="913.00281" y1="647.5" x1="1040" - gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-55.792546,-371.61471)" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-13.745322,-248.93757)" gradientUnits="userSpaceOnUse" id="linearGradient3293-66" xlink:href="#linearGradient5177-5-7-0-0-94" @@ -1396,6 +1396,98 @@ style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" transform="scale(0.8,0.8)" /> </marker> + <linearGradient + y2="527.5" + x2="940" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,0.90036935,-201.53597)" + gradientUnits="userSpaceOnUse" + id="linearGradient3171-0" + xlink:href="#linearGradient5177-5-7-0-3" + inkscape:collect="always" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-5-7-0-3"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-4-5-6-49" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-3-4-7-5" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-30" + style="overflow:visible"> + <path + id="path4855-6" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="527.5" + x2="940" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,42.947594,-50.512371)" + gradientUnits="userSpaceOnUse" + id="linearGradient3345" + xlink:href="#linearGradient5177-5-7-0-3" + inkscape:collect="always" /> + <linearGradient + y2="507.75555" + x2="913.00281" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,15.073596,-215.7092)" + gradientUnits="userSpaceOnUse" + id="linearGradient3293-9-9" + xlink:href="#linearGradient5177-5-7-0-0-3-2" + inkscape:collect="always" /> + <linearGradient + inkscape:collect="always" + id="linearGradient5177-5-7-0-0-3-2"> + <stop + style="stop-color:#ffffff;stop-opacity:1;" + offset="0" + id="stop5179-4-5-6-6-00-5" /> + <stop + style="stop-color:#ffffff;stop-opacity:0;" + offset="1" + id="stop5181-3-4-7-1-7-7" /> + </linearGradient> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0" + refX="0" + id="TriangleOutL-24" + style="overflow:visible"> + <path + id="path4855-162" + d="m 5.77,0 -8.65,5 0,-10 8.65,5 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="scale(0.8,0.8)" + inkscape:connector-curvature="0" /> + </marker> + <linearGradient + y2="507.75555" + x2="913.00281" + y1="647.5" + x1="1040" + gradientTransform="matrix(0.99477643,-0.10207768,0.10207768,0.99477643,-141.30438,-22.165921)" + gradientUnits="userSpaceOnUse" + id="linearGradient3385" + xlink:href="#linearGradient5177-5-7-0-0-3-2" + inkscape:collect="always" /> </defs> <sodipodi:namedview id="base" @@ -1404,14 +1496,14 @@ borderopacity="1.0" inkscape:pageopacity="0.0" inkscape:pageshadow="2" - inkscape:zoom="0.6253155" - inkscape:cx="1260.4791" - inkscape:cy="468.32395" + inkscape:zoom="0.6883404" + inkscape:cx="816.49415" + inkscape:cy="515.86812" inkscape:document-units="px" - inkscape:current-layer="g3346" + inkscape:current-layer="layer1" showgrid="true" - inkscape:window-width="1920" - inkscape:window-height="1176" + inkscape:window-width="1600" + inkscape:window-height="876" inkscape:window-x="0" inkscape:window-y="24" inkscape:window-maximized="1" @@ -1444,7 +1536,7 @@ <dc:format>image/svg+xml</dc:format> <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> - <dc:title /> + <dc:title></dc:title> </cc:Work> </rdf:RDF> </metadata> @@ -1452,28 +1544,28 @@ inkscape:label="Layer 1" inkscape:groupmode="layer" id="layer1" - transform="translate(42.5,62.637817)"> + transform="translate(42.5,-57.362183)"> <rect style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:3.17159414;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.17159461, 3.17159461;stroke-dashoffset:0" id="rect5533" - width="1600" - height="376.22046" - x="-27.854328" - y="407.75592" + width="1601.1024" + height="335.27554" + x="-28.326771" + y="530.43304" rx="43.243244" ry="14.887338" /> <rect - style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:2.31244493;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:2.31244524, 2.31244524;stroke-dashoffset:0" + style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:2.57646298;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:2.57646335, 2.57646335;stroke-dashoffset:0" id="rect5533-7" - width="1600" - height="200" - x="-27.854328" - y="883.97638" + width="1600.4725" + height="141.7323" + x="-23.759842" + y="944.76373" rx="43.243244" - ry="7.914156" /> + ry="9.824481" /> <g id="g3260" - transform="translate(54.645669,326.61418)"> + transform="translate(59.212598,400.7874)"> <rect ry="33.33334" rx="40" @@ -1501,7 +1593,7 @@ </g> <g id="g3296" - transform="translate(-285.35433,446.61418)"> + transform="translate(-280.7874,520.7874)"> <rect ry="33.33334" rx="40" @@ -1521,11 +1613,11 @@ y="443.03146" x="877.81494" sodipodi:role="line" - id="tspan3304">clock</tspan></text> + id="tspan3304">timer</tspan></text> </g> <g id="g3352" - transform="translate(-325.35433,366.61418)"> + transform="translate(-320.7874,440.7874)"> <rect ry="33.33334" rx="40" @@ -1553,7 +1645,7 @@ </g> <g id="g3346" - transform="translate(-221.25984,232.44095)"> + transform="translate(-216.69291,306.61417)"> <rect inkscape:transform-center-y="83.33334" inkscape:transform-center-x="-80" @@ -1583,7 +1675,7 @@ </g> <path style="fill:url(#linearGradient5183);fill-opacity:1;stroke:#000000;stroke-width:2.16024685;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 223.95669,550.52754 56.69292,0" + d="m 266.00391,673.20468 56.69292,0" id="path4299" inkscape:connector-type="polyline" inkscape:connector-curvature="0" @@ -1591,38 +1683,38 @@ <text xml:space="preserve" style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" - x="792.14569" - y="983.97638" + x="796.71265" + y="1058.1497" id="text3983" sodipodi:linespacing="125%"><tspan sodipodi:role="line" id="tspan3985" - x="792.14569" - y="983.97638" + x="796.71265" + y="1058.1497" style="text-align:center;text-anchor:middle">System interfaces</tspan><tspan sodipodi:role="line" - x="792.14569" - y="1033.9764" + x="796.71265" + y="1108.1497" id="tspan3989" - style="text-align:center;text-anchor:middle">(exposed by Aesop functions)</tspan><tspan + style="text-align:center;text-anchor:middle" /><tspan sodipodi:role="line" - x="792.14569" - y="1083.9764" + x="796.71265" + y="1158.1497" id="tspan3987" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7" - y="564.70081" - x="119.75189" + y="687.37793" + x="161.79912" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="564.70081" - x="124.74408" + y="687.37793" + x="166.79131" sodipodi:role="line" id="tspan4293-3" style="font-weight:bold">{ state:1 } </tspan><tspan - y="609.70081" - x="119.75189" + y="732.37793" + x="161.79912" sodipodi:role="line" id="tspan4295-5" /></text> <flowRoot @@ -1638,167 +1730,167 @@ id="flowPara4024" /></flowRoot> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-0" - y="564.70081" - x="389.04324" + y="687.37793" + x="431.09045" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="564.70081" - x="394.03543" + y="687.37793" + x="436.08264" sodipodi:role="line" id="tspan4293-3-0" style="font-weight:bold">{ state:2 } </tspan><tspan - y="609.70081" - x="389.04324" + y="732.37793" + x="431.09045" sodipodi:role="line" id="tspan4295-5-0" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-5" - y="479.97638" - x="747.1535" + y="596.4173" + x="788.25586" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="479.97638" - x="752.14569" + y="596.4173" + x="793.24805" sodipodi:role="line" id="tspan4293-3-4" style="font-weight:bold">{ state:3[0] } </tspan><tspan - y="524.97638" - x="747.1535" + y="641.4173" + x="788.25586" sodipodi:role="line" id="tspan4295-5-1" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-4" - y="559.97638" - x="747.1535" + y="653.11017" + x="788.25586" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="559.97638" - x="752.14569" + y="653.11017" + x="793.24805" sodipodi:role="line" id="tspan4293-3-1" style="font-weight:bold">{ state:3[1] } </tspan><tspan - y="604.97638" - x="747.1535" + y="698.11017" + x="788.25586" sodipodi:role="line" id="tspan4295-5-7" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-2" - y="639.97638" - x="747.1535" + y="709.8031" + x="788.25586" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="639.97638" - x="752.14569" + y="709.8031" + x="793.24805" sodipodi:role="line" id="tspan4293-3-03" style="font-weight:bold">{ state:3[2] } </tspan><tspan - y="684.97638" - x="747.1535" + y="754.8031" + x="788.25586" sodipodi:role="line" id="tspan4295-5-2" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-49" - y="564.70081" - x="1026.8385" + y="687.37793" + x="1068.8857" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="564.70081" - x="1031.8307" + y="687.37793" + x="1073.8779" sodipodi:role="line" id="tspan4293-3-8" style="font-weight:bold">{ state:4 } </tspan><tspan - y="609.70081" - x="1026.8385" + y="732.37793" + x="1068.8857" sodipodi:role="line" id="tspan4295-5-19" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-3" - y="499.97638" - x="1387.1534" + y="638.93695" + x="1355.1849" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="499.97638" - x="1392.1456" + y="638.93695" + x="1360.1771" sodipodi:role="line" id="tspan4293-3-9" style="font-weight:bold">{ state:5a } </tspan><tspan - y="544.97638" - x="1387.1534" + y="683.93695" + x="1355.1849" sodipodi:role="line" id="tspan4295-5-5" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-9" - y="619.97638" - x="1387.1534" + y="738.14954" + x="1355.1849" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="619.97638" - x="1392.1456" + y="738.14954" + x="1360.1771" sodipodi:role="line" id="tspan4293-3-10" style="font-weight:bold">{ state:5b } </tspan><tspan - y="664.97638" - x="1387.1534" + y="783.14954" + x="1355.1849" sodipodi:role="line" id="tspan4295-5-3" /></text> <path style="fill:url(#linearGradient3171);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="M 861.75197,621.39369 918.44488,578.874" + d="m 899.54722,702.71649 63.77953,-14.17323" id="path4299-7-2-0" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3211-5);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 861.75197,479.6614 56.69291,42.51969" + d="m 899.54722,596.41728 70.86614,49.60629" id="path4299-7-2-0-0" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 861.75197,550.52754 56.69291,0" + d="m 899.54722,646.02357 63.77953,21.25985" id="path4299-7-2-0-0-8" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-9);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 479.0748,550.52754 141.73229,56.69292" + d="m 523.95667,674.37003 141.73228,21.25984" id="path4299-7-2-0-0-80" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-2);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 479.0748,550.52754 141.73229,0" + d="M 523.95667,674.37003 665.68895,653.11019" id="path4299-7-2-0-0-1" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-8);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 1116.87,564.70077 155.9055,28.34646" + d="m 1162.2244,672.16533 84.567,37.63777" id="path4299-7-2-0-0-17" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-1);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="m 1116.87,536.35432 155.9055,-28.34646" + d="m 1162.2244,672.16533 84.567,-33.22837" id="path4299-7-2-0-0-4" inkscape:connector-type="polyline" inkscape:connector-curvature="0" sodipodi:nodetypes="cc" /> <path style="fill:url(#linearGradient3293-66);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" - d="M 479.0748,550.52754 620.80709,479.6614" + d="M 523.95667,674.37003 662.85431,602.33854" id="path4299-7-2-0-0-47" inkscape:connector-type="polyline" inkscape:connector-curvature="0" @@ -1806,23 +1898,18 @@ <text xml:space="preserve" style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" - x="790.88586" - y="705.39368" + x="790.41345" + y="828.0708" id="text3983-3" sodipodi:linespacing="125%"><tspan sodipodi:role="line" - id="tspan3985-7" - x="790.88586" - y="705.39368" - style="text-align:center;text-anchor:middle">Poll-compatible states and transitions</tspan><tspan - sodipodi:role="line" - x="790.88586" - y="755.39368" + x="790.41345" + y="828.0708" id="tspan3989-7" - style="text-align:center;text-anchor:middle">(generated by Aesop compiler)</tspan><tspan + style="text-align:center;text-anchor:middle">States and transitions</tspan><tspan sodipodi:role="line" - x="790.88586" - y="805.39368" + x="790.41345" + y="878.0708" id="tspan3987-5" /></text> <rect style="fill:#fcfcfc;fill-opacity:1;stroke:#000000;stroke-width:3.37171936;stroke-linecap:round;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:3.37171988, 3.37171988;stroke-dashoffset:0" @@ -1830,142 +1917,158 @@ width="1600" height="425.19687" x="-28.326771" - y="-50.669312" + y="76.889725" rx="43.243244" ry="16.825371" /> <text xml:space="preserve" style="font-size:40.75279617px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" - x="31.19434" - y="5.9123359" + x="14.460023" + y="131.11508" id="text3844" sodipodi:linespacing="125%" transform="scale(0.98152775,1.0188199)"><tspan sodipodi:role="line" id="tspan3846" - x="31.19434" - y="5.9123359" + x="14.460023" + y="131.11508" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan style="font-weight:bold" - id="tspan3982">aesop service_function</tspan>() { </tspan><tspan + id="tspan3982"><tspan + style="font-weight:normal;-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3412">1</tspan> aesop service_function</tspan>() { </tspan><tspan sodipodi:role="line" - x="31.19434" - y="50.912338" + x="14.460023" + y="176.11508" id="tspan3848" - style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"> char data[MAX]; </tspan><tspan + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3418">2</tspan> char data[MAX]; </tspan><tspan sodipodi:role="line" - x="31.19434" - y="95.912338" + x="14.460023" + y="221.11508" id="tspan3850" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan style="font-weight:bold" - id="tspan4004"> network_operation</tspan>();</tspan><tspan + id="tspan4004"><tspan + style="font-weight:normal;-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3414">3</tspan> network_operation</tspan>();</tspan><tspan sodipodi:role="line" - x="31.19434" - y="140.91232" + x="14.460023" + y="266.11508" id="tspan3852" - style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"> for ( ... ) { </tspan><tspan + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3422">4</tspan> for ( ... ) { </tspan><tspan sodipodi:role="line" - x="31.19434" - y="185.91232" + x="14.460023" + y="311.11508" id="tspan3854" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan style="font-weight:bold" - id="tspan4002"> disk_operation</tspan>();</tspan><tspan + id="tspan4002"><tspan + style="font-weight:normal;-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3410">5</tspan> disk_operation</tspan>();</tspan><tspan sodipodi:role="line" - x="31.19434" - y="230.91234" + x="14.460023" + y="356.11508" id="tspan3856" - style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"> }</tspan><tspan + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3426">6</tspan> }</tspan><tspan sodipodi:role="line" - x="31.19434" - y="275.91235" + x="14.460023" + y="401.11508" id="tspan3858" - style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"> if (<tspan + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3428">7</tspan> if (<tspan style="font-weight:bold" id="tspan4000">db_operation</tspan>()) { ... } </tspan><tspan sodipodi:role="line" - x="31.19434" - y="320.91235" + x="14.460023" + y="446.11508" id="tspan3860" - style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"> else { ... } </tspan></text> + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:start;line-height:125%;writing-mode:lr-tb;text-anchor:start;font-family:DejaVu Sans Mono;-inkscape-font-specification:DejaVu Sans Mono"><tspan + style="-inkscape-font-specification:Courier 10 Pitch;font-family:Courier 10 Pitch;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal" + id="tspan3430">8</tspan> else { ... } </tspan></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-37" - y="48.543289" + y="176.10233" x="829.67316" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="48.543289" + y="176.10233" x="834.66534" sodipodi:role="line" id="tspan4293-3-3" style="font-weight:bold">{ state:1 } </tspan><tspan - y="93.543289" + y="221.10233" x="829.67316" sodipodi:role="line" id="tspan4295-5-4" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-0-6" - y="133.58264" + y="261.14166" x="829.67316" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="133.58264" + y="261.14166" x="834.66534" sodipodi:role="line" id="tspan4293-3-0-4" style="font-weight:bold">{ state:2 } </tspan><tspan - y="178.58264" + y="306.14166" x="829.67316" sodipodi:role="line" id="tspan4295-5-0-4" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-5-3" - y="190.27557" + y="317.83459" x="829.67316" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="190.27557" + y="317.83459" x="834.66534" sodipodi:role="line" id="tspan4293-3-4-7" - style="font-weight:bold">{ state:3[] } </tspan><tspan - y="235.27557" + style="font-weight:bold">{ state:3[...] } </tspan><tspan + y="362.83459" x="829.67316" sodipodi:role="line" id="tspan4295-5-1-7" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-5-3-5" - y="246.96849" + y="374.52753" x="829.67316" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="246.96849" + y="374.52753" x="834.66534" sodipodi:role="line" id="tspan4293-3-4-7-7" style="font-weight:bold">{ state:4 } </tspan><tspan - y="291.96851" + y="419.52753" x="829.67316" sodipodi:role="line" id="tspan4295-5-1-7-5" /></text> <text sodipodi:linespacing="125%" id="text2985-5-2-8-5-7-3-7" - y="303.66141" + y="431.22043" x="829.67316" style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" xml:space="preserve"><tspan - y="303.66141" + y="431.22043" x="834.66534" sodipodi:role="line" id="tspan4293-3-9-7" style="font-weight:bold">{ state:5a,b } </tspan><tspan - y="348.66141" + y="476.22043" x="829.67316" sodipodi:role="line" id="tspan4295-5-5-7" /></text> @@ -1973,28 +2076,64 @@ xml:space="preserve" style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" x="990.57092" - y="91.062973" + y="218.62201" id="text3983-3-8" sodipodi:linespacing="125%"><tspan sodipodi:role="line" x="990.57092" - y="91.062973" + y="218.62201" id="tspan3987-5-3">Non-blocking operations </tspan><tspan sodipodi:role="line" x="990.57092" - y="141.06297" + y="268.62201" id="tspan4042">(shown in bold) and </tspan><tspan sodipodi:role="line" x="990.57092" - y="191.06297" + y="318.62201" id="tspan4048">service code logic (states)</tspan><tspan sodipodi:role="line" x="990.57092" - y="241.06297" + y="368.62201" id="tspan4044">expressed as sequential </tspan><tspan sodipodi:role="line" x="990.57092" - y="291.06299" + y="418.62201" id="tspan4040">control flow </tspan></text> + <text + sodipodi:linespacing="125%" + id="text2985-5-2-8-5-7-2-9" + y="766.49597" + x="788.25586" + style="font-size:36px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial" + xml:space="preserve"><tspan + y="766.49597" + x="793.24805" + sodipodi:role="line" + id="tspan4293-3-03-1" + style="font-weight:bold">{ state:3[...] } </tspan><tspan + y="811.49597" + x="788.25586" + sodipodi:role="line" + id="tspan4295-5-2-1" /></text> + <path + style="fill:url(#linearGradient3345);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" + d="M 906.63383,752.32279 970.41336,709.8031" + id="path4299-7-2-0-6" + inkscape:connector-type="polyline" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:url(#linearGradient3385);fill-opacity:1;stroke:#000000;stroke-width:2;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;marker-start:none;marker-end:url(#TriangleOutL);display:inline" + d="m 523.95667,674.37003 141.73228,70.86614" + id="path4299-7-2-0-0-80-5" + inkscape:connector-type="polyline" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" + d="m 56.712598,105.2362 c 0,396.85039 0,382.67716 0,382.67716" + id="path3432" + inkscape:connector-curvature="0" + sodipodi:nodetypes="cc" /> </g> </svg> hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. fb54078b154c8bbedff6ab9f47d235036e8fa789
by noreply＠mcs.anl.gov 07 Mar '12

07 Mar '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via fb54078b154c8bbedff6ab9f47d235036e8fa789 (commit) via c4fbf88e257cfbc7955dd1e4b9c194ad06be0215 (commit) via f76d05ea8364df7071b895e8bcdee45264a3e2a4 (commit) from 4a747ecc6126e254734f23b0f9df28410cf243f4 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit fb54078b154c8bbedff6ab9f47d235036e8fa789 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Mar 7 15:58:30 2012 -0600 Example use of script2tex for discussion commit c4fbf88e257cfbc7955dd1e4b9c194ad06be0215 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Mar 7 15:57:45 2012 -0600 Bring in script2tex commit f76d05ea8364df7071b895e8bcdee45264a3e2a4 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Mar 7 15:47:38 2012 -0600 Address epoll comment ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/context.tex | 24 +++++--- papers/2011/aesop/listings/listing-1.ae | 22 +++++++ papers/2011/aesop/listings/listing-1.ae.tex | 32 ++++++++++ papers/2011/aesop/model.tex | 56 +++++++++--------- papers/2011/aesop/script2tex.pl | 84 +++++++++++++++++++++++++++ 5 files changed, 182 insertions(+), 36 deletions(-) create mode 100644 papers/2011/aesop/listings/listing-1.ae create mode 100644 papers/2011/aesop/listings/listing-1.ae.tex create mode 100755 papers/2011/aesop/script2tex.pl Diff of changes: diff --git a/papers/2011/aesop/context.tex b/papers/2011/aesop/context.tex index cf945af..819ac16 100644 --- a/papers/2011/aesop/context.tex +++ b/papers/2011/aesop/context.tex @@ -248,22 +248,28 @@ send/receive and one-sided put/get communication. MPI-2~\cite{gropp1999ump,snir1995mcr} offers non-blocking messaging calls for point-to-point communication. -\color{red}PHC: the following paragraph needs some work. You can't actually -use select, poll, or epoll on files in practice. Maybe just cut all of that -and just point out the POSIX AIO stuff? There are other async disk I/O -libraries, but none of them are widely used and portable so I don't know -that they are worth mentioning.\color{black} - -Disk operations also have a wide range of non-blocking access methods. Unix systems have traditionally offered the select/poll paradigm, in which file descriptors are collected into a data structure; the waiting process then makes a library call that does not return until some file descriptor makes progress (or an exception occurs). epoll~\cite{EPOLL_WWW}, available in Linux, offers a similar model but is typically more scalable for large numbers of file descriptors. + +%% \color{red}PHC: the following paragraph needs some work. You can't +%% actually use select, poll, or epoll on files in practice. Maybe +%% just cut all of that and just point out the POSIX AIO stuff? There +%% are other async disk I/O libraries, but none of them are widely +%% used and portable so I don't know that they are worth +%% mentioning.\color{black} + +%% Addressed. -Justin + +Disk operations also have a range of non-blocking access methods. POSIX AIO~\cite{POSIX_AIO_2006} offers multiple models for controlling -asynchronous I/O, including the ability to supply a signal or -callback function for completion notification. +asynchronous I/O, including the ability to supply a signal or callback +function for completion notification. Additionally, the user may +manually place disk operations in separate threads to enhance +concurrency. Databases are commonly used for file system metadata and the efficient use of them is critical. Berkeley~DB~\cite{BDB_1999} does not provide diff --git a/papers/2011/aesop/listings/listing-1.ae b/papers/2011/aesop/listings/listing-1.ae new file mode 100644 index 0000000..0a7977f --- /dev/null +++ b/papers/2011/aesop/listings/listing-1.ae @@ -0,0 +1,22 @@ +aesop void doWrite (...) { + pwait { + pbranch { + aesop_timer (TIMEOUT); + ae_cancel_branches (); + } + pbranch { + replicateWrite (...); + ae_cancel_branches (); + } + } +} + +aesop void replicateWrite (...) { + pwait { + pprivate int i; + for (i=0; i<REPLICAS; ++i) + pbranch { + replicateWriteTo (i); + } + } +} diff --git a/papers/2011/aesop/listings/listing-1.ae.tex b/papers/2011/aesop/listings/listing-1.ae.tex new file mode 100644 index 0000000..238a1b7 --- /dev/null +++ b/papers/2011/aesop/listings/listing-1.ae.tex @@ -0,0 +1,32 @@ + +\begin{center} +\scriptsize + +\fbox{ +\begin{tabular}{r|l} + 1 & {\tt aesop\ void\ doWrite\ (...)\ \{ } \\ + 2 & {\tt \ \ pwait\ \{ } \\ + 3 & {\tt \ \ \ \ pbranch\ \{ } \\ + 4 & {\tt \ \ \ \ \ \ aesop\_timer\ (TIMEOUT); } \\ + 5 & {\tt \ \ \ \ \ \ ae\_cancel\_branches\ (); } \\ + 6 & {\tt \ \ \ \ \} } \\ + 7 & {\tt \ \ \ \ pbranch\ \{ } \\ + 8 & {\tt \ \ \ \ \ \ replicateWrite\ (...); } \\ + 9 & {\tt \ \ \ \ \ \ ae\_cancel\_branches\ (); } \\ +10 & {\tt \ \ \ \ \} } \\ +11 & {\tt \ \ \} } \\ +12 & {\tt \} } \\ +13 & {\tt } \\ +14 & {\tt aesop\ void\ replicateWrite\ (...)\ \{ } \\ +15 & {\tt \ \ pwait\ \{ } \\ +16 & {\tt \ \ \ \ pprivate\ int\ i; } \\ +17 & {\tt \ \ \ \ for\ (i=0;\ i<REPLICAS;\ ++i) } \\ +18 & {\tt \ \ \ \ \ \ pbranch\ \{ } \\ +19 & {\tt \ \ \ \ \ \ \ \ replicateWriteTo\ (i); } \\ +20 & {\tt \ \ \ \ \ \ \} } \\ +21 & {\tt \ \ \} } \\ +22 & {\tt \} } \\ + +\end{tabular} +} +\end{center} diff --git a/papers/2011/aesop/model.tex b/papers/2011/aesop/model.tex index 4787319..e477df5 100644 --- a/papers/2011/aesop/model.tex +++ b/papers/2011/aesop/model.tex @@ -135,36 +135,38 @@ to a request, cancelling the request when the allocated time is exceeded. +%% \begin{figure} +%% \begin{lstlisting} +%% __blocking void doWrite (...) { +%% pwait { +%% pbranch { +%% aesop_timer (TIMEOUT); +%% ae_cancel_branches (); +%% } +%% pbranch { +%% replicateWrite (...); +%% ae_cancel_branches (); +%% } +%% } +%% } +%% __blocking void replicateWrite (...) { +%% pwait { +%% pprivate int i; +%% for (i=0; i<REPLICAS; ++i) +%% pbranch { +%% replicateWriteTo (i); +%% } +%% } +%% } +%% \end{lstlisting} +%% \caption{Aesop Code Example} +%% \end{figure} + \begin{figure} - \begin{lstlisting} -__blocking void doWrite (...) { - pwait { - pbranch { - aesop_timer (TIMEOUT); - ae_cancel_branches (); - } - pbranch { - replicateWrite (...); - ae_cancel_branches (); - } - } -} -__blocking void replicateWrite (...) { - pwait { - pprivate int i; - for (i=0; i<REPLICAS; ++i) - pbranch { - replicateWriteTo (i); - } - } -} -\end{lstlisting} -\caption{Aesop Code Example} +\input listings/listing-1.ae.tex +\caption{Aesop example code.} \end{figure} - - - \color{red} Somewhere (not sure where in the paper) we should address how aesop uses multiple cores. Right now, normal c code may or may not be executed in a diff --git a/papers/2011/aesop/script2tex.pl b/papers/2011/aesop/script2tex.pl new file mode 100755 index 0000000..20ef069 --- /dev/null +++ b/papers/2011/aesop/script2tex.pl @@ -0,0 +1,84 @@ +#!/usr/bin/perl -n + +# Process code for insertion into LaTeX so verbatim is not required + +# example usage: script2tex.pl file.c > file.c.tex + +# Makes a nice line-numbered table +# Uses \tt +# Escapes curly braces +# Escapes percent sign (%) +# Allows for bold sections +# Use _B_ to get {\bf (BOLD) +# Use _C_ to get } (CLOSE BRACE) +# Escapes underscores +# Escapes spaces (for indentation) +# Puts C++ comments in textrm +# Puts shell (hash) comments in textrm +# Use "-Example- 32" to get a label + +# If NO_BOX is non-empty in the environment, suppress the fbox + +BEGIN { + $N = 1; + $EXAMPLE = -1; + $BOX = 1; + if ($ENV{NO_BOX} != "") { + $BOX = 0; + } + + print "\n"; + print "\\begin{center}\n"; + print "\\scriptsize\n\n"; + if ($BOX == 1) { + print "\\fbox{\n"; + } + print "\\begin{tabular}{r|l}\n"; +} + +{ + chomp; + if (/.*-Example- (.*)/) { + $EXAMPLE = $1; + } + else { + printf("%2i & ", $N); + printf("{\\tt "); + s/{/\\{/g; + s/}/\\}/g; + s/%/\\%/g; + s{//(.*)}{// \\textrm{$1}}g; + s{#(.*)}{\\# \\textrm{$1}}g; + s/ /\\ /g; + s/_B_/{\\bf /g; + s/_C_/}/g; + s/_/\\_/g; + printf("%-40s", $_); + printf("} "); + printf("\\\\\n"); + $N++; + } +} + +END { + print "\n"; + if ($EXAMPLE != -1) { + print "\\hline\n"; + } + print "\\end{tabular}\n"; + if ($BOX == 1) { + print "}\n"; + } + if ($EXAMPLE != -1) { + print "\\\\\n"; + print "\\vspace{1mm}\n"; + print "\\textbf{Example $EXAMPLE}\n"; + } + print "\\end{center}\n"; +} + +# Local Variables: +# indent-tabs-mode: f +# tab-width: 2 +# perl-indent-level: 2 +# End: hooks/post-receive -- Triton-private Repository

1 0