October 2018 - Ssio-2018 - mailman3.cels.anl.gov

Re: [Ssio-2018] reminder of homework
by Ross, Robert B. 30 Oct '18

30 Oct '18

We were hoping to have a complete draft Nov. 9. I understand the situation. Do what you can. -- Rob On Oct 30, 2018, at 12:40 PM, Glenn Lockwood via ssio-2018 <ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]>> wrote: What is the intended deadline for having all of this done? I am a bit under water right now with SC and facilities stuff, but if it needs to be done before SC I can shuffle obligations around. Glenn On Tue, Oct 30, 2018 at 10:33 AM Ross, Robert B. via ssio-2018 <ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]>> wrote: The workshop discussion is primarily meant to be more comprehensive than the distilled PRDs. I think what you have is ok, although you might want to add an introductory paragraph that just says the discussion followed a different organization than we had originally envisioned, or something like that? rob On Oct 30, 2018, at 12:11 PM, Mohror, Kathryn <mohror1(a)llnl.gov<mailto:[email protected]>> wrote: Hi Rob, Thanks for this. Somehow I didn’t get that the workshop discussion and the promising research directions were two separate sections. In my mind the outcome of the workshop discussion *was* the promising research directions so they were the same. Is the workshop discussion supposed to be more of just notes on the discussion (in whatever order they occurred) and the PRD supposed to be distilled and organized as we did in the slides we made recently? Thanks, Kathryn From: ssio-2018 <ssio-2018-bounces(a)lists.mcs.anl.gov<mailto:[email protected]>> On Behalf Of Ross, Robert B. via ssio-2018 Sent: Tuesday, October 30, 2018 8:37 AM To: ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> Subject: [Ssio-2018] reminder of homework Hi all, To recap, there are (at least) three things that we all ought to be making sure are done right now: 1. Writing up a "Workshop Discussion" sub subsection for whatever sections you're assigned to. 2. Writing up a "Promising Research Directions" sub subsection (following the above) for whatever sections you're assigned to. 3. Writing up a narrative in the PRD slides that captures some additional detail, beyond the material in the slide. (2) and (3) are related. Here are some quick notes on how Phil and I accomplished writing the (1) Workshop Discussion sub subsection for the Understanding subsection. Feel free to do your own thing, but we thought this might be helpful: a. Made a copy of the notes from the discussion at the workshop, cut out the intro stuff. b. Noted that the discussion followed the pre-workshop organization, so followed that in the new sub subsection. c. Split up the areas, wrote up 0.5-1.0 pages of narrative for each (e.g., Modeling Systems), with an inline header for each in the sub subsection. Marked off items in the (copy of) the notes as we went, as a sanity check that we got everything written down. d. Added an additional paragraph for "Facilitating Research", which covers the testbed and data availability topics. We're going to move on to the (2) PRD subsubsection next. We plan to write that first, then do (3). Best, Rob and Phil _______________________________________________ ssio-2018 mailing list ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/ssio-2018 _______________________________________________ ssio-2018 mailing list ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/ssio-2018

1 0

Re: [Ssio-2018] reminder of homework
by Glenn Lockwood 30 Oct '18

30 Oct '18

What is the intended deadline for having all of this done? I am a bit under water right now with SC and facilities stuff, but if it needs to be done before SC I can shuffle obligations around. Glenn On Tue, Oct 30, 2018 at 10:33 AM Ross, Robert B. via ssio-2018 < ssio-2018(a)lists.mcs.anl.gov> wrote: > The workshop discussion is primarily meant to be more comprehensive than > the distilled PRDs. I think what you have is ok, although you might want to > add an introductory paragraph that just says the discussion followed a > different organization than we had originally envisioned, or something like > that? > > rob > > On Oct 30, 2018, at 12:11 PM, Mohror, Kathryn <mohror1(a)llnl.gov> wrote: > > Hi Rob, > > Thanks for this. Somehow I didn’t get that the workshop discussion and the > promising research directions were two separate sections. In my mind the > outcome of the workshop discussion **was** the promising research > directions so they were the same. > > Is the workshop discussion supposed to be more of just notes on the > discussion (in whatever order they occurred) and the PRD supposed to be > distilled and organized as we did in the slides we made recently? > > Thanks, > Kathryn > > *From:* ssio-2018 <ssio-2018-bounces(a)lists.mcs.anl.gov> *On Behalf Of *Ross, > Robert B. via ssio-2018 > *Sent:* Tuesday, October 30, 2018 8:37 AM > *To:* ssio-2018(a)lists.mcs.anl.gov > *Subject:* [Ssio-2018] reminder of homework > > Hi all, > > To recap, there are (at least) three things that we all ought to be making > sure are done right now: > 1. Writing up a "Workshop Discussion" sub subsection for whatever sections > you're assigned to. > 2. Writing up a "Promising Research Directions" sub subsection (following > the above) for whatever sections you're assigned to. > 3. Writing up a narrative in the PRD slides that captures some additional > detail, beyond the material in the slide. (2) and (3) are related. > > Here are some quick notes on how Phil and I accomplished writing the (1) > Workshop Discussion sub subsection for the Understanding subsection. Feel > free to do your own thing, but we thought this might be helpful: > a. Made a copy of the notes from the discussion at the workshop, cut out > the intro stuff. > b. Noted that the discussion followed the pre-workshop organization, so > followed that in the new sub subsection. > c. Split up the areas, wrote up 0.5-1.0 pages of narrative for each (e.g., > Modeling Systems), with an inline header for each in the sub subsection. > Marked off items in the (copy of) the notes as we went, as a sanity check > that we got everything written down. > d. Added an additional paragraph for "Facilitating Research", which covers > the testbed and data availability topics. > > We're going to move on to the (2) PRD subsubsection next. We plan to write > that first, then do (3). > > Best, > > Rob and Phil > > > _______________________________________________ > ssio-2018 mailing list > ssio-2018(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/ssio-2018 >

1 0

Re: [Ssio-2018] reminder of homework
by Ross, Robert B. 30 Oct '18

30 Oct '18

The workshop discussion is primarily meant to be more comprehensive than the distilled PRDs. I think what you have is ok, although you might want to add an introductory paragraph that just says the discussion followed a different organization than we had originally envisioned, or something like that? rob On Oct 30, 2018, at 12:11 PM, Mohror, Kathryn <mohror1(a)llnl.gov<mailto:[email protected]>> wrote: Hi Rob, Thanks for this. Somehow I didn’t get that the workshop discussion and the promising research directions were two separate sections. In my mind the outcome of the workshop discussion *was* the promising research directions so they were the same. Is the workshop discussion supposed to be more of just notes on the discussion (in whatever order they occurred) and the PRD supposed to be distilled and organized as we did in the slides we made recently? Thanks, Kathryn From: ssio-2018 <ssio-2018-bounces(a)lists.mcs.anl.gov<mailto:[email protected]>> On Behalf Of Ross, Robert B. via ssio-2018 Sent: Tuesday, October 30, 2018 8:37 AM To: ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> Subject: [Ssio-2018] reminder of homework Hi all, To recap, there are (at least) three things that we all ought to be making sure are done right now: 1. Writing up a "Workshop Discussion" sub subsection for whatever sections you're assigned to. 2. Writing up a "Promising Research Directions" sub subsection (following the above) for whatever sections you're assigned to. 3. Writing up a narrative in the PRD slides that captures some additional detail, beyond the material in the slide. (2) and (3) are related. Here are some quick notes on how Phil and I accomplished writing the (1) Workshop Discussion sub subsection for the Understanding subsection. Feel free to do your own thing, but we thought this might be helpful: a. Made a copy of the notes from the discussion at the workshop, cut out the intro stuff. b. Noted that the discussion followed the pre-workshop organization, so followed that in the new sub subsection. c. Split up the areas, wrote up 0.5-1.0 pages of narrative for each (e.g., Modeling Systems), with an inline header for each in the sub subsection. Marked off items in the (copy of) the notes as we went, as a sanity check that we got everything written down. d. Added an additional paragraph for "Facilitating Research", which covers the testbed and data availability topics. We're going to move on to the (2) PRD subsubsection next. We plan to write that first, then do (3). Best, Rob and Phil

1 0

Re: [Ssio-2018] reminder of homework
by Mohror, Kathryn 30 Oct '18

30 Oct '18

Hi Rob, Thanks for this. Somehow I didn't get that the workshop discussion and the promising research directions were two separate sections. In my mind the outcome of the workshop discussion *was* the promising research directions so they were the same. Is the workshop discussion supposed to be more of just notes on the discussion (in whatever order they occurred) and the PRD supposed to be distilled and organized as we did in the slides we made recently? Thanks, Kathryn From: ssio-2018 <ssio-2018-bounces(a)lists.mcs.anl.gov> On Behalf Of Ross, Robert B. via ssio-2018 Sent: Tuesday, October 30, 2018 8:37 AM To: ssio-2018(a)lists.mcs.anl.gov Subject: [Ssio-2018] reminder of homework Hi all, To recap, there are (at least) three things that we all ought to be making sure are done right now: 1. Writing up a "Workshop Discussion" sub subsection for whatever sections you're assigned to. 2. Writing up a "Promising Research Directions" sub subsection (following the above) for whatever sections you're assigned to. 3. Writing up a narrative in the PRD slides that captures some additional detail, beyond the material in the slide. (2) and (3) are related. Here are some quick notes on how Phil and I accomplished writing the (1) Workshop Discussion sub subsection for the Understanding subsection. Feel free to do your own thing, but we thought this might be helpful: a. Made a copy of the notes from the discussion at the workshop, cut out the intro stuff. b. Noted that the discussion followed the pre-workshop organization, so followed that in the new sub subsection. c. Split up the areas, wrote up 0.5-1.0 pages of narrative for each (e.g., Modeling Systems), with an inline header for each in the sub subsection. Marked off items in the (copy of) the notes as we went, as a sanity check that we got everything written down. d. Added an additional paragraph for "Facilitating Research", which covers the testbed and data availability topics. We're going to move on to the (2) PRD subsubsection next. We plan to write that first, then do (3). Best, Rob and Phil

1 0

Re: [Ssio-2018] PRD narrative example for discussion
by Ross, Robert B. 30 Oct '18

30 Oct '18

Wow, the formatting was a lot better when I wrote it. Hopefully you get the idea. -- Rob On Oct 30, 2018, at 11:55 AM, Ross, Robert B. via ssio-2018 <ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]>> wrote: Hi all, Phil and I have been working on the PRD narrative for the Understanding section. I've attached our draft version below. The approach we took was the following: 1. Copy in all the material from the corresponding PRD slide. 2. First paragraph gets lightly edited into an intro. 3. Top-level bullets become bolded topics. 4. Sub-bullets become bullets to those. 5. Added some narrative below that provides a little additional material/detail on those topics. Lucy, all: Does this look about right as an example? Thanks! Rob and Phil --- 5.5.6 Promising Research Directions Participants concluded that better understanding of SSIO systems leads to more productive applications, enhanced adaptation of running services, and more effective design, development, and procurement of tomorrow’s applications, systems, and system software. The following priorities emerged from the discussion. Enabling real-time and post hoc analysis through instrumentation, capture, and retention of monitoring data. * Scalable and minimally intrusive methods of data gathering on application- and system-side, keeping pace with technological advancements in hardware and software * Streaming delivery of data for real-time analysis and decision-making Scalable and minimally intrusive data gathering methods enable the investigation of full-scale applications and systems rather than just experimental examples. This ensures that instrumentation data is representative of real-world behavior, and also enables real time feedback to users. For such instrumentation to remain relevant, it must incorporate emerging technologies as they are deployed. This includes new hardware such as NVRAM, new programming models such as machine learning frameworks, and new architectures such as object storage systems. Furthermore, streaming delivery of data will make the it applicable not just for post-hoc analysis, but for real time analysis and real time control feedback. Predicting behavior through workload, software stack, and architectural modeling. * Analytical, ML, and reduced models for rapid decision-making * Validated models at multiple fidelities for design space exploration * System models to aid in data integration Modeling of relevant systems, services and workloads complement data gathering activities. A variety of validated, predictive models should be considered depending on the use case for the model. This calls for further research in methods including analytical, machine learning, and reduced models to cover the spectrum of high fidelity and rapid decision-making use cases. Additionally, integration of SSIO models with models capturing other aspects of the system (e.g., network and communication workload, scheduler) must be accounted for when tackling questions of importance to facilities, both in terms of day-to-day operations and future procurement decisions. Integrating data sources and identifying correlations to improve our understanding. * Real-time and post hoc data fusion and learning * Requires availability of large data sets for learning applications Transformation of monitoring data into knowledge is a nontrivial task that naturally involves fusion of data from multiple sources. Important real-time (e.g., autonomics) and post hoc (e.g., performance tuning) use cases exist with distinct needs. Machine learning is a promising approach to knowledge generation, but for these approaches to be effective, the aforementioned challenges in instrumentation and capture must be addressed in order to gather the requisite data on which learning can occur. _______________________________________________ ssio-2018 mailing list ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/ssio-2018

1 0

PRD narrative example for discussion
by Ross, Robert B. 30 Oct '18

30 Oct '18

Hi all, Phil and I have been working on the PRD narrative for the Understanding section. I've attached our draft version below. The approach we took was the following: 1. Copy in all the material from the corresponding PRD slide. 2. First paragraph gets lightly edited into an intro. 3. Top-level bullets become bolded topics. 4. Sub-bullets become bullets to those. 5. Added some narrative below that provides a little additional material/detail on those topics. Lucy, all: Does this look about right as an example? Thanks! Rob and Phil --- 5.5.6 Promising Research Directions Participants concluded that better understanding of SSIO systems leads to more productive applications, enhanced adaptation of running services, and more effective design, development, and procurement of tomorrow’s applications, systems, and system software. The following priorities emerged from the discussion. Enabling real-time and post hoc analysis through instrumentation, capture, and retention of monitoring data. * Scalable and minimally intrusive methods of data gathering on application- and system-side, keeping pace with technological advancements in hardware and software * Streaming delivery of data for real-time analysis and decision-making Scalable and minimally intrusive data gathering methods enable the investigation of full-scale applications and systems rather than just experimental examples. This ensures that instrumentation data is representative of real-world behavior, and also enables real time feedback to users. For such instrumentation to remain relevant, it must incorporate emerging technologies as they are deployed. This includes new hardware such as NVRAM, new programming models such as machine learning frameworks, and new architectures such as object storage systems. Furthermore, streaming delivery of data will make the it applicable not just for post-hoc analysis, but for real time analysis and real time control feedback. Predicting behavior through workload, software stack, and architectural modeling. * Analytical, ML, and reduced models for rapid decision-making * Validated models at multiple fidelities for design space exploration * System models to aid in data integration Modeling of relevant systems, services and workloads complement data gathering activities. A variety of validated, predictive models should be considered depending on the use case for the model. This calls for further research in methods including analytical, machine learning, and reduced models to cover the spectrum of high fidelity and rapid decision-making use cases. Additionally, integration of SSIO models with models capturing other aspects of the system (e.g., network and communication workload, scheduler) must be accounted for when tackling questions of importance to facilities, both in terms of day-to-day operations and future procurement decisions. Integrating data sources and identifying correlations to improve our understanding. * Real-time and post hoc data fusion and learning * Requires availability of large data sets for learning applications Transformation of monitoring data into knowledge is a nontrivial task that naturally involves fusion of data from multiple sources. Important real-time (e.g., autonomics) and post hoc (e.g., performance tuning) use cases exist with distinct needs. Machine learning is a promising approach to knowledge generation, but for these approaches to be effective, the aforementioned challenges in instrumentation and capture must be addressed in order to gather the requisite data on which learning can occur.

1 0

reminder of homework
by Ross, Robert B. 30 Oct '18

30 Oct '18

Hi all, To recap, there are (at least) three things that we all ought to be making sure are done right now: 1. Writing up a "Workshop Discussion" sub subsection for whatever sections you're assigned to. 2. Writing up a "Promising Research Directions" sub subsection (following the above) for whatever sections you're assigned to. 3. Writing up a narrative in the PRD slides that captures some additional detail, beyond the material in the slide. (2) and (3) are related. Here are some quick notes on how Phil and I accomplished writing the (1) Workshop Discussion sub subsection for the Understanding subsection. Feel free to do your own thing, but we thought this might be helpful: a. Made a copy of the notes from the discussion at the workshop, cut out the intro stuff. b. Noted that the discussion followed the pre-workshop organization, so followed that in the new sub subsection. c. Split up the areas, wrote up 0.5-1.0 pages of narrative for each (e.g., Modeling Systems), with an inline header for each in the sub subsection. Marked off items in the (copy of) the notes as we went, as a sanity check that we got everything written down. d. Added an additional paragraph for "Facilitating Research", which covers the testbed and data availability topics. We're going to move on to the (2) PRD subsubsection next. We plan to write that first, then do (3). Best, Rob and Phil

1 0

Re: [Ssio-2018] Workshop discussion section
by Mohror, Kathryn 30 Oct '18

30 Oct '18

Hi Rob, Thanks. For my section, most of the content was included in the PRD, but there were a few minor points that I added to the discussion. I also added a bit more detail and tech words that weren't appropriate in the PRD. Kathryn From: Ross, Robert B. <rross(a)mcs.anl.gov> Sent: Monday, October 29, 2018 12:15 PM To: Mohror, Kathryn <mohror1(a)llnl.gov> Cc: ssio-2018(a)lists.mcs.anl.gov Subject: Re: [Ssio-2018] Workshop discussion section Hi Kathryn, This sounds good. One clarification though; it's possible that there was discussion during the workshop that you want to include in the Workshop Discussion section but that didn't percolate up to the PRD level. It's ok to have that also. rob On Oct 29, 2018, at 12:51 PM, Mohror, Kathryn <mohror1(a)llnl.gov<mailto:[email protected]>> wrote: Hi all, I made a draft of the workshop discussion section for Section 5. Can you let me know if it is what you are looking for? Based on what I understood the plan to be from our discussion at the SSIO meeting, I made a new subsection at the end of 5 called Workshop Discussion and put text there that follows the main points in the summary slides we made. Thanks, Kathryn _________________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov<mailto:[email protected]>, https://people.llnl.gov/kathryn Data Analysis Group<https://computation.llnl.gov/casc/data-analysis-group> @ Lawrence Livermore National Laboratory, Livermore, CA, USA _______________________________________________ ssio-2018 mailing list ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/ssio-2018

1 0

Re: [Ssio-2018] Workshop discussion section
by Ross, Robert B. 29 Oct '18

29 Oct '18

Hi Kathryn, This sounds good. One clarification though; it's possible that there was discussion during the workshop that you want to include in the Workshop Discussion section but that didn't percolate up to the PRD level. It's ok to have that also. rob On Oct 29, 2018, at 12:51 PM, Mohror, Kathryn <mohror1(a)llnl.gov<mailto:[email protected]>> wrote: Hi all, I made a draft of the workshop discussion section for Section 5. Can you let me know if it is what you are looking for? Based on what I understood the plan to be from our discussion at the SSIO meeting, I made a new subsection at the end of 5 called Workshop Discussion and put text there that follows the main points in the summary slides we made. Thanks, Kathryn _________________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov<mailto:[email protected]>, https://people.llnl.gov/kathryn Data Analysis Group<https://computation.llnl.gov/casc/data-analysis-group> @ Lawrence Livermore National Laboratory, Livermore, CA, USA _______________________________________________ ssio-2018 mailing list ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> https://lists.mcs.anl.gov/mailman/listinfo/ssio-2018

1 0

Re: [Ssio-2018] Workshop discussion section
by Mohror, Kathryn 29 Oct '18

29 Oct '18

Hi Scott, I think that is the plan of attack based on the notes I found here: https://docs.google.com/document/d/1IdM851XdbgzG14xN7VKWv1oZmpC6_7BxqNRBIrh… I was checking in with the group to make sure I understood the plan correctly. Perhaps Rob or Lee will chime in to let us know shortly. Kathryn From: Klasky, Scott A. <klasky(a)ornl.gov> Sent: Monday, October 29, 2018 11:36 AM To: Mohror, Kathryn <mohror1(a)llnl.gov>; ssio-2018(a)lists.mcs.anl.gov Subject: RE: Workshop discussion section Hi Kathryn, Is this the plan of attack for finishing up the document? I will work with Matt on my section I the very near future to follow this guideline . Cheers, Scott From: ssio-2018 <ssio-2018-bounces(a)lists.mcs.anl.gov<mailto:[email protected]>> On Behalf Of Mohror, Kathryn Sent: Monday, October 29, 2018 1:52 PM To: ssio-2018(a)lists.mcs.anl.gov<mailto:[email protected]> Subject: [Ssio-2018] Workshop discussion section Hi all, I made a draft of the workshop discussion section for Section 5. Can you let me know if it is what you are looking for? Based on what I understood the plan to be from our discussion at the SSIO meeting, I made a new subsection at the end of 5 called Workshop Discussion and put text there that follows the main points in the summary slides we made. Thanks, Kathryn _________________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov<mailto:[email protected]>, https://people.llnl.gov/kathryn Data Analysis Group<https://computation.llnl.gov/casc/data-analysis-group> @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0