January 2013 - Release-team - mailman3.cels.anl.gov

Minor release?
by Nomi Harris 30 Jan '13

30 Jan '13

Did we officially decide not to do a minor release based on last week's mini-build, and just to wait for the full build/release? Nomi

1 0

Re: [Release-team] phispy module failed deploy
by Land, Miriam L. 30 Jan '13

30 Jan '13

I think it would be a good item to add to "Information Relevant to Builds", https://trac.kbase.us/projects/kbase/wiki/BuildProcess#InformationRelevantt oBuilds". The ports themselves are in the middle of "Integration Targets", https://trac.kbase.us/projects/kbase/wiki/IntegrationTargets. Along those same lines, can we add information or links to this Build page on where to put tutorials and user manuals? I just had someone in my office who wanted to know. Miriam On 1/30/13 12:29 PM, "Nomi Harris" <nlharris(a)lbl.gov> wrote: >They probably didn't realize you had to register a port for the service. >Is this in any of our documentation? If not, it should be added. >I wonder if it should be mentioned on Miriam's new Release Week schedule >(https://trac.kbase.us/projects/kbase/wiki/ReleaseWeek) or if it's too >fine-grained for that. > Nomi > >On Jan 28, 2013, at 9:51 AM, Dan Olson <dolson(a)mcs.anl.gov> wrote: > >> The phispy module failed deployment because there is no registered port >>for the service >> at https://trac.kbase.us/projects/kbase/wiki/IntegrationTargets. >> >> ---- >> Daniel Murphy-Olson >> Systems Administrator >> Mathematics & Computer Science Division >> Argonne National Laboratory >> 630-252-0055 > >_______________________________________________ >Release-team mailing list >Release-team(a)lists.kbase.us >https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] phispy module failed deploy
by Nomi Harris 30 Jan '13

30 Jan '13

They probably didn't realize you had to register a port for the service. Is this in any of our documentation? If not, it should be added. I wonder if it should be mentioned on Miriam's new Release Week schedule (https://trac.kbase.us/projects/kbase/wiki/ReleaseWeek) or if it's too fine-grained for that. Nomi On Jan 28, 2013, at 9:51 AM, Dan Olson <dolson(a)mcs.anl.gov> wrote: > The phispy module failed deployment because there is no registered port for the service > at https://trac.kbase.us/projects/kbase/wiki/IntegrationTargets. > > ---- > Daniel Murphy-Olson > Systems Administrator > Mathematics & Computer Science Division > Argonne National Laboratory > 630-252-0055

1 0

Release week schedule
by Land, Miriam L. 30 Jan '13

30 Jan '13

All, I have created a wiki page for the release week schedule (https://trac.kbase.us/projects/kbase/wiki/ReleaseWeek) I needed it so I could reference it in emails rather than repeating the information in the emails themselves. As always, any of you can edit, update, correct, expand, or otherwise improve. The only link to it at the moment is from the 'Information Relevant to Builds' (https://trac.kbase.us/projects/kbase/wiki/BuildProcess) Miriam

1 0

Re: [Release-team] Deployed into Production - testing
by Land, Miriam L. 29 Jan '13

29 Jan '13

Shane, I've looked into testing the production version of these. As you said, kb_seed and idserver didn't need redeploying. The workspace and cluster_service have encountered other problems since they were last tested. They should be resolved with this next release. The erdb_service is a dependency of other modules and should be private. It would be hard to test this without logging into the production server. That leaves communities_qc. The response I get back from http://kbase.us/services/communities_qc has changed during the time I have tried to test this. I'll work with the developer to see what tests are appropriate. Miriam On 1/28/13 6:32 PM, "Shane Canon" <scanon(a)lbl.gov> wrote: > >Cary and I have deployed the following modules into production. All of >these used the tag 01222013-test. > >communities_qc >erdb_service >kb_seed >workspace >idserver >cluster_service > >I think several of these didn't need redeploying (i.e. kb_seed and >idserver) since they were only tested because they were dependencies. > >I tried running make test on a few and ran into some issues. I will >contact the maintainers and ask them to test the production deployment. > >--Shane > >_______________________________________________ >Release-team mailing list >Release-team(a)lists.kbase.us >https://lists.kbase.us/mailman/listinfo/release-team

1 0

modules passed
by Dan Olson 29 Jan '13

29 Jan '13

Deployment took longer than expected yesterday. I've completed the deployment attempt on all the targeted modules, the dependancies I'm still working on. I'm also still working on the documentation. The following modules passed: KBaseFBAModeling matR sim_service translation ---- Daniel Murphy-Olson Systems Administrator Mathematics & Computer Science Division Argonne National Laboratory 630-252-0055

1 0

Re: [Release-team] [Kbase-devel] [FEB] what is required for all command line scripts before feb demo
by Shiran Pasternak 28 Jan '13

28 Jan '13

On Jan 28, 2013, at 3:05 PM, Nomi Harris <nlharris(a)lbl.gov> wrote: > I added brief text and a link to https://trac.kbase.us/projects/kbase/wiki/UserManuals explaining that if you follow the standard for your help message, then your help text will be harvested as part of the release process and made into a man page for your script. Right now, that's at the bottom of the User Manuals page--should it be made more prominent? Are we going to discourage people from hand-writing user manuals and encourage them to write good help messages instead? I still think that a user manual should be developed manually. Command help messages are just that. They don't necessarily give a good overview or an indication of where to start. User manuals can be a lot more interactive, primarily through hypertext, and can also be used to give uninitiated users or those uncomfortable with running Unix commands in the first place a better place to start. > Yes, we need that. Maybe you could put up an initial straw-man page for that, Shiran? I'll draft that up tomorrow as time allows. Bob, it would be great if you can review it to make sure I don't misspeak. > Agreed. Thanks for being proactive about this. Are there other discussions that we should put on the wiki? Good question. The discussion on standardizing RPC exceptions comes to mind. Maybe we should have a wiki page for that. I'll do that to when I get the chance. Shiran > > Nomi > >> >> Shiran >> >> [1] (cf. emails from last week I can't link to because this is not a wiki) >> >> On Jan 25, 2013, at 12:56 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >> >>> Hi everyone. >>> >>> In the note from Adam on Wednesday, this statement is made: >>> >>>> Command line scripts. All command line scripts need to be accessible from IRIS. If you are not creating command line scripts for your module, we need to know this and why. Tom will talk on the devel con call today what is required for integration into IRIS. >>> >>> In order for us to accomplish this, module owners will need to ensure the following: >>> >>> Script Documentation >>> 1) that all commands return zero when invoked with the -h and the --help options. >>> 2) that all commands print to standard out a help message when invoked with the -h and --help options. >>> 3) that the output contains the following information (see template below) >>> a description of what the script does >>> a description of the parameters >>> a description of the inputs and outputs >>> >>> We'll harvest this text as part of the release process (see below) >>> >>> Naming Convention >>> We will be cleaning up the namespace soon as a single, comprehensive effort in the near future. To help, you can start now by following this simple guidance: >>> >>> 1) commands are named lowercase with a consistent use of underscores (or dashes). >>> 2) the kb prefix will be dropped in future releases >>> 3) prefix with a module identifier >>> >>> examples are : >>> >>> Auxiliary Store: >>> as_create_index as_list_nodes as_resources >>> as_create_node as_list_users as_view_node >>> as_create_user as_modify_node as_view_user >>> >>> Workspace Service (drop the kb): >>> kbws-createws kbws-load kbws-types.pl >>> kbws-delete kbws-login kbws-url >>> kbws-deletews kbws-logout kbws-whoami >>> kbws-exists kbws-meta kbws-workspace >>> >>> Testing >>> The use of the test-scripts target will become a mandatory requirement for scripts that will be executed within IRIS. Because we will have hundreds of commands, we need an automated way to at a minimum verify that the script will run (this verifies the deployment was done correctly and all runtime dependancies are in place). As a minimal start, you should write a shell script that. >>> >>> 1) invokes the script using pre-staged test data >>> 2) verifies correct output. >>> 3) verifies the correct exit code >>> 4) one directory per test >>> 5) this directory contains the input data and output to be compared against by way of a diff or something of your choice. >>> 6) these test directories are located under the top level script-tests directory as per the standard laid out in the module.Makefile template. >>> >>> Complex Commands >>> Commands with complex interfaces, such as jnomics and the assembly service needs to be wrapped into simpler scripts for the user that set preset parameter values and turn the command/subcommand style interface into just a command interface. >>> >>> Commands running on the IRIS host >>> Those commands that themselves are not clients in a client-server architecture, that is commands that will execute on the IRIS host (matR comes to mind), need to have communicated to the release engineering team cpu, memory and storage requirements. >>> >>> Release Process >>> What I will do is modify the release process so that all commands are checked for the -h and --help options. The process will go something like first check to see if the -h and --help options exist and the command returns 0, for those that pass this check we'll have someone take a look at the output to see if it addresses the items in #3. The output will be saved as plain text, moved to the webroot directory under the deployment target, and from there be brought into the kbase.us/developer_zone site as html docs. >>> >>> >>> >>> Here is the template: >>> >>> NAME >>> commandname -- one line summary of what it does >>> >>> SYNOPSIS >>> commandname [list of options] >>> >>> DESCRIPTION >>> Few sentences to paragraphs describing what the command does >>> >>> A detailed description of the inputs >>> >>> A detailed description of the outputs >>> >>> name of parameter 1 - description of parameter (one parameter per line) >>> >>> name of parameter 2 - description of parameter 2 (etc.) >>> >>> EXAMPLES >>> An example or two of how to call the command and what it would do >>> >>> SEE ALSO >>> List any related commands >>> >>> AUTHORS >>> Make yourself (and your coauthors) famous >>> >>> >>> >>> NAME >>> kbws-meta -- displays the metadata for the currently selected workspace >>> >>> SYNOPSIS >>> kbws-meta [-we] >>> >>> DESCRIPTION >>> Workspaces are used in KBase to provide an online location for all data, models, and analysis results. kbws-meta displays the metadata (for example, the date last modified and the permission settings) of the currently selected workspace (or the workspace specified on the command line with the "-w/--workspace" option). >>> >>> Inputs: none >>> >>> Outputs: metadata fields and values printed to stdout, one per line >>> >>> Options: >>> -w,--workspace ID of workspace you want to look at >>> -e,--showerror Set to 1 to show any errors in execution >>> >>> EXAMPLES >>> $ kbws-meta >>> Workspace ID: test >>> Owner: chenry >>> Moddate: 2012-12-26T06:11:50 >>> Objects: 4 >>> User permission: r >>> Global permission:r >>> >>> SEE ALSO >>> kbws-list >>> kbws-workspace >>> kbws-listobj >>> >>> AUTHORS >>> Chris Henry >>> >>> Thomas Brettin >>> 865-603-7760 (cell) >>> brettin(a)mcs.anl.gov >

1 0

Deployed into Production
by Shane Canon 28 Jan '13

28 Jan '13

Cary and I have deployed the following modules into production. All of these used the tag 01222013-test. communities_qc erdb_service kb_seed workspace idserver cluster_service I think several of these didn't need redeploying (i.e. kb_seed and idserver) since they were only tested because they were dependencies. I tried running make test on a few and ran into some issues. I will contact the maintainers and ask them to test the production deployment. --Shane

1 0

Re: [Release-team] [Kbase-devel] [FEB] what is required for all command line scripts before feb demo
by Nomi Harris 28 Jan '13

28 Jan '13

I also added a link to the CLI Standards page on https://trac.kbase.us/projects/kbase/wiki/DocStandards Nomi On Jan 28, 2013, at 12:05 PM, Nomi Harris <nlharris(a)lbl.gov> wrote: > On Jan 28, 2013, at 7:13 AM, Shiran Pasternak <shiranpasternak(a)gmail.com> wrote: > >> Hi guys, >> >> I've incorporated Tom's email into the Trac Wiki page on command-line standards, started a few months ago by Paramvir. This should ensure that such important documentation on SOP and decisions don't get lost in our brimming inboxes. > > Thanks, Shiran. > I added brief text and a link to https://trac.kbase.us/projects/kbase/wiki/UserManuals explaining that if you follow the standard for your help message, then your help text will be harvested as part of the release process and made into a man page for your script. Right now, that's at the bottom of the User Manuals page--should it be made more prominent? Are we going to discourage people from hand-writing user manuals and encourage them to write good help messages instead? > >> Still missing from the Trac Wiki: SOP on exposing commands in IRIS.[1] > > Yes, we need that. Maybe you could put up an initial straw-man page for that, Shiran? > >> Documentation on the Trac Wiki has been improving dramatically in the past few months, but it's still very sparse relative to all the discussions happening on this list. > > Agreed. Thanks for being proactive about this. Are there other discussions that we should put on the wiki? > > Nomi > >> >> Shiran >> >> [1] (cf. emails from last week I can't link to because this is not a wiki) >> >> On Jan 25, 2013, at 12:56 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >> >>> Hi everyone. >>> >>> In the note from Adam on Wednesday, this statement is made: >>> >>>> Command line scripts. All command line scripts need to be accessible from IRIS. If you are not creating command line scripts for your module, we need to know this and why. Tom will talk on the devel con call today what is required for integration into IRIS. >>> >>> In order for us to accomplish this, module owners will need to ensure the following: >>> >>> Script Documentation >>> 1) that all commands return zero when invoked with the -h and the --help options. >>> 2) that all commands print to standard out a help message when invoked with the -h and --help options. >>> 3) that the output contains the following information (see template below) >>> a description of what the script does >>> a description of the parameters >>> a description of the inputs and outputs >>> >>> We'll harvest this text as part of the release process (see below) >>> >>> Naming Convention >>> We will be cleaning up the namespace soon as a single, comprehensive effort in the near future. To help, you can start now by following this simple guidance: >>> >>> 1) commands are named lowercase with a consistent use of underscores (or dashes). >>> 2) the kb prefix will be dropped in future releases >>> 3) prefix with a module identifier >>> >>> examples are : >>> >>> Auxiliary Store: >>> as_create_index as_list_nodes as_resources >>> as_create_node as_list_users as_view_node >>> as_create_user as_modify_node as_view_user >>> >>> Workspace Service (drop the kb): >>> kbws-createws kbws-load kbws-types.pl >>> kbws-delete kbws-login kbws-url >>> kbws-deletews kbws-logout kbws-whoami >>> kbws-exists kbws-meta kbws-workspace >>> >>> Testing >>> The use of the test-scripts target will become a mandatory requirement for scripts that will be executed within IRIS. Because we will have hundreds of commands, we need an automated way to at a minimum verify that the script will run (this verifies the deployment was done correctly and all runtime dependancies are in place). As a minimal start, you should write a shell script that. >>> >>> 1) invokes the script using pre-staged test data >>> 2) verifies correct output. >>> 3) verifies the correct exit code >>> 4) one directory per test >>> 5) this directory contains the input data and output to be compared against by way of a diff or something of your choice. >>> 6) these test directories are located under the top level script-tests directory as per the standard laid out in the module.Makefile template. >>> >>> Complex Commands >>> Commands with complex interfaces, such as jnomics and the assembly service needs to be wrapped into simpler scripts for the user that set preset parameter values and turn the command/subcommand style interface into just a command interface. >>> >>> Commands running on the IRIS host >>> Those commands that themselves are not clients in a client-server architecture, that is commands that will execute on the IRIS host (matR comes to mind), need to have communicated to the release engineering team cpu, memory and storage requirements. >>> >>> Release Process >>> What I will do is modify the release process so that all commands are checked for the -h and --help options. The process will go something like first check to see if the -h and --help options exist and the command returns 0, for those that pass this check we'll have someone take a look at the output to see if it addresses the items in #3. The output will be saved as plain text, moved to the webroot directory under the deployment target, and from there be brought into the kbase.us/developer_zone site as html docs. >>> >>> >>> >>> Here is the template: >>> >>> NAME >>> commandname -- one line summary of what it does >>> >>> SYNOPSIS >>> commandname [list of options] >>> >>> DESCRIPTION >>> Few sentences to paragraphs describing what the command does >>> >>> A detailed description of the inputs >>> >>> A detailed description of the outputs >>> >>> name of parameter 1 - description of parameter (one parameter per line) >>> >>> name of parameter 2 - description of parameter 2 (etc.) >>> >>> EXAMPLES >>> An example or two of how to call the command and what it would do >>> >>> SEE ALSO >>> List any related commands >>> >>> AUTHORS >>> Make yourself (and your coauthors) famous >>> >>> >>> >>> NAME >>> kbws-meta -- displays the metadata for the currently selected workspace >>> >>> SYNOPSIS >>> kbws-meta [-we] >>> >>> DESCRIPTION >>> Workspaces are used in KBase to provide an online location for all data, models, and analysis results. kbws-meta displays the metadata (for example, the date last modified and the permission settings) of the currently selected workspace (or the workspace specified on the command line with the "-w/--workspace" option). >>> >>> Inputs: none >>> >>> Outputs: metadata fields and values printed to stdout, one per line >>> >>> Options: >>> -w,--workspace ID of workspace you want to look at >>> -e,--showerror Set to 1 to show any errors in execution >>> >>> EXAMPLES >>> $ kbws-meta >>> Workspace ID: test >>> Owner: chenry >>> Moddate: 2012-12-26T06:11:50 >>> Objects: 4 >>> User permission: r >>> Global permission:r >>> >>> SEE ALSO >>> kbws-list >>> kbws-workspace >>> kbws-listobj >>> >>> AUTHORS >>> Chris Henry >>> >>> Thomas Brettin >>> 865-603-7760 (cell) >>> brettin(a)mcs.anl.gov >

1 0

Re: [Release-team] [Kbase-devel] [FEB] what is required for all command line scripts before feb demo
by Nomi Harris 28 Jan '13

28 Jan '13

On Jan 28, 2013, at 7:13 AM, Shiran Pasternak <shiranpasternak(a)gmail.com> wrote: > Hi guys, > > I've incorporated Tom's email into the Trac Wiki page on command-line standards, started a few months ago by Paramvir. This should ensure that such important documentation on SOP and decisions don't get lost in our brimming inboxes. Thanks, Shiran. I added brief text and a link to https://trac.kbase.us/projects/kbase/wiki/UserManuals explaining that if you follow the standard for your help message, then your help text will be harvested as part of the release process and made into a man page for your script. Right now, that's at the bottom of the User Manuals page--should it be made more prominent? Are we going to discourage people from hand-writing user manuals and encourage them to write good help messages instead? > Still missing from the Trac Wiki: SOP on exposing commands in IRIS.[1] Yes, we need that. Maybe you could put up an initial straw-man page for that, Shiran? > Documentation on the Trac Wiki has been improving dramatically in the past few months, but it's still very sparse relative to all the discussions happening on this list. Agreed. Thanks for being proactive about this. Are there other discussions that we should put on the wiki? Nomi > > Shiran > > [1] (cf. emails from last week I can't link to because this is not a wiki) > > On Jan 25, 2013, at 12:56 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > >> Hi everyone. >> >> In the note from Adam on Wednesday, this statement is made: >> >>> Command line scripts. All command line scripts need to be accessible from IRIS. If you are not creating command line scripts for your module, we need to know this and why. Tom will talk on the devel con call today what is required for integration into IRIS. >> >> In order for us to accomplish this, module owners will need to ensure the following: >> >> Script Documentation >> 1) that all commands return zero when invoked with the -h and the --help options. >> 2) that all commands print to standard out a help message when invoked with the -h and --help options. >> 3) that the output contains the following information (see template below) >> a description of what the script does >> a description of the parameters >> a description of the inputs and outputs >> >> We'll harvest this text as part of the release process (see below) >> >> Naming Convention >> We will be cleaning up the namespace soon as a single, comprehensive effort in the near future. To help, you can start now by following this simple guidance: >> >> 1) commands are named lowercase with a consistent use of underscores (or dashes). >> 2) the kb prefix will be dropped in future releases >> 3) prefix with a module identifier >> >> examples are : >> >> Auxiliary Store: >> as_create_index as_list_nodes as_resources >> as_create_node as_list_users as_view_node >> as_create_user as_modify_node as_view_user >> >> Workspace Service (drop the kb): >> kbws-createws kbws-load kbws-types.pl >> kbws-delete kbws-login kbws-url >> kbws-deletews kbws-logout kbws-whoami >> kbws-exists kbws-meta kbws-workspace >> >> Testing >> The use of the test-scripts target will become a mandatory requirement for scripts that will be executed within IRIS. Because we will have hundreds of commands, we need an automated way to at a minimum verify that the script will run (this verifies the deployment was done correctly and all runtime dependancies are in place). As a minimal start, you should write a shell script that. >> >> 1) invokes the script using pre-staged test data >> 2) verifies correct output. >> 3) verifies the correct exit code >> 4) one directory per test >> 5) this directory contains the input data and output to be compared against by way of a diff or something of your choice. >> 6) these test directories are located under the top level script-tests directory as per the standard laid out in the module.Makefile template. >> >> Complex Commands >> Commands with complex interfaces, such as jnomics and the assembly service needs to be wrapped into simpler scripts for the user that set preset parameter values and turn the command/subcommand style interface into just a command interface. >> >> Commands running on the IRIS host >> Those commands that themselves are not clients in a client-server architecture, that is commands that will execute on the IRIS host (matR comes to mind), need to have communicated to the release engineering team cpu, memory and storage requirements. >> >> Release Process >> What I will do is modify the release process so that all commands are checked for the -h and --help options. The process will go something like first check to see if the -h and --help options exist and the command returns 0, for those that pass this check we'll have someone take a look at the output to see if it addresses the items in #3. The output will be saved as plain text, moved to the webroot directory under the deployment target, and from there be brought into the kbase.us/developer_zone site as html docs. >> >> >> >> Here is the template: >> >> NAME >> commandname -- one line summary of what it does >> >> SYNOPSIS >> commandname [list of options] >> >> DESCRIPTION >> Few sentences to paragraphs describing what the command does >> >> A detailed description of the inputs >> >> A detailed description of the outputs >> >> name of parameter 1 - description of parameter (one parameter per line) >> >> name of parameter 2 - description of parameter 2 (etc.) >> >> EXAMPLES >> An example or two of how to call the command and what it would do >> >> SEE ALSO >> List any related commands >> >> AUTHORS >> Make yourself (and your coauthors) famous >> >> >> >> NAME >> kbws-meta -- displays the metadata for the currently selected workspace >> >> SYNOPSIS >> kbws-meta [-we] >> >> DESCRIPTION >> Workspaces are used in KBase to provide an online location for all data, models, and analysis results. kbws-meta displays the metadata (for example, the date last modified and the permission settings) of the currently selected workspace (or the workspace specified on the command line with the "-w/--workspace" option). >> >> Inputs: none >> >> Outputs: metadata fields and values printed to stdout, one per line >> >> Options: >> -w,--workspace ID of workspace you want to look at >> -e,--showerror Set to 1 to show any errors in execution >> >> EXAMPLES >> $ kbws-meta >> Workspace ID: test >> Owner: chenry >> Moddate: 2012-12-26T06:11:50 >> Objects: 4 >> User permission: r >> Global permission:r >> >> SEE ALSO >> kbws-list >> kbws-workspace >> kbws-listobj >> >> AUTHORS >> Chris Henry >> >> Thomas Brettin >> 865-603-7760 (cell) >> brettin(a)mcs.anl.gov

1 0