July 2013 - Release-team - mailman3.cels.anl.gov

Re: [Release-team] Expression Candidate for release
by Robert Olson 10 Jul '13

10 Jul '13

Have you been working with Bruce on the plan for doing an integration with the production XML for the CDM ? On Jul 10, 2013, at 1:30 PM, Jason Baumohl wrote: > Hi All, > > I am not opposed to creating some scripts for the command line. I have not done this before, but I suspect it is essentially a wrapper to functions in the service. > > As far as the autogenerated functions (get_entity_*.pl, get_relationship_*.pl, and query_entity_*.pl) > I do have those for my db tables in > root@db1-expression:/kb/dev_container/modules/kb_seed/scripts# > > -Jason > > > > On Wed, Jul 10, 2013 at 9:50 AM, Robert Olson <olson(a)mcs.anl.gov> wrote: > If the expression stuff is actually integrated into the central store data model, it'll have the get_entity/get_rel/etc scripts created as a matter of course. > > If it's being maintained as a separate database, those will need to be separately generated (compile_dbd_to_typespec & friends). > > It seems we should have some command line scripts to do things like enumerate experiments that are relevant to a given genome ID; probes that match a given set of features, retrieval of normalized data for a given experiment, etc. > > --bob > > On Jul 9, 2013, at 5:45 PM, Thomas Brettin wrote: > >> You should talk to Bob if you're wondering about automatically generated scripts. He, Ross and Terry also wrote many hand crafted scripts as well. >> >> The poster child I sent you (registry) was for the module Makefile, but it also includes commands. Additionally, it contains a template file that I used to auto generate the registry commands (see scripts/script.tt and scripts/tpage_command), but you should really talk to Bob about this as his auto generate skills are superior to mine. >> >> Tom >> >> >> >> On Jul 9, 2013, at 5:37 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> >>> Hi Tom, >>> >>> Tom you are welcome to play around with my make file. >>> Yes, this would save me some time, but I have not made scripts and commands to work in IRIS before. >>> That is why I was asking about a poster child example. So I really do not know what is involved. >>> Currently there are over 30 (and growing) functions that I have made for expression. >>> If it is only a wrapper then it should be ok. The automated relationship and entity functions do not go intp the scripts section. I believe these get made automatically when I run the CDMI stuff on mmy XML. >>> The potential issue there is that expression is in a different database. >>> >>> -Jason >>> >>> >>> >>> >>> On Tue, Jul 9, 2013 at 2:26 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >>> I haven't enforced a policy on the script requirement yet, although it seems having scripts for standard services, particularly those in the CDMI space could be required. >>> >>> I do enforce a policy that all modules must have a COMMANDS file, but it is not unheard of that someone will just create an empty file in their module. >>> >>> Jason, if I were to take on cleaning up the make file for the release would you then have time to get some commands in place before we release. Also, is it possible to automate the generation of the Entity and Relations type of commands? I'm sure Bob didn't hand write all those. >>> >>> Tom >>> >>> >>> On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: >>> >>>> For the initial release we should have command line and iris I would think in addition to whatever we have in the API? >>>> >>>> We have several others working on pipelines for RNAseq analysis and I think we need to harmonize these before too long. >>>> >>>> On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >>>> >>>>> Rick asked for the commands. >>>>> >>>>> >>>>> On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>> >>>>>> I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. >>>>>> >>>>>> -Jason >>>>>> >>>>>> >>>>>> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>>> Hi Tom, >>>>>> >>>>>> I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? >>>>>> >>>>>> Thanks, >>>>>> Jason >>>>>> >>>>>> >>>>>> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>>> Hi Jason, >>>>>> >>>>>> I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. >>>>>> >>>>>> As an example, you have in your make file: >>>>>> >>>>>> deploy-service-scripts: >>>>>> # First create the start script (should be a better way to do this…) >>>>>> … >>>>>> >>>>>> Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. >>>>>> >>>>>> Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? >>>>>> >>>>>> Thanks, >>>>>> Tom >>>>>> >>>>>> >>>>>> >>>>>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>>> >>>>>>> Works for me. >>>>>>> >>>>>>> -Jason >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>>>> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >>>>>>> >>>>>>> >>>>>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>>>> >>>>>>> > Hi Tom and Paramvir, >>>>>>> > >>>>>>> > I think I have an official candidate for release for expression. >>>>>>> > >>>>>>> > I have a bunch of stuff but includes the following: >>>>>>> > spec file >>>>>>> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >>>>>>> > automated testing for all completed functions >>>>>>> > readme >>>>>>> > releasenotes >>>>>>> > exchangeformat specs >>>>>>> > uploader >>>>>>> > preprocessors >>>>>>> > initial data loaded into expression on DB1 >>>>>>> > automated doc for Impl code >>>>>>> > documentation for exchange format (file format for uploads). >>>>>>> > >>>>>>> > I think I have everything except scripts and commands to be run from the command line and iris. >>>>>>> > >>>>>>> > All of the code is committed to git. >>>>>>> > >>>>>>> > -Jason >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Release-team mailing list >>>>>> Release-team(a)lists.kbase.us >>>>>> https://lists.kbase.us/mailman/listinfo/release-team >>>>> >>>> >>> >>> >> > >

1 0

Re: [Release-team] Expression Candidate for release
by Jason Baumohl 10 Jul '13

10 Jul '13

Hi All, I am not opposed to creating some scripts for the command line. I have not done this before, but I suspect it is essentially a wrapper to functions in the service. As far as the autogenerated functions (get_entity_*.pl, get_relationship_*.pl, and query_entity_*.pl) I do have those for my db tables in root@db1-expression:/kb/dev_container/modules/kb_seed/scripts# -Jason On Wed, Jul 10, 2013 at 9:50 AM, Robert Olson <olson(a)mcs.anl.gov> wrote: > If the expression stuff is actually integrated into the central store data > model, it'll have the get_entity/get_rel/etc scripts created as a matter of > course. > > If it's being maintained as a separate database, those will need to be > separately generated (compile_dbd_to_typespec & friends). > > It seems we should have some command line scripts to do things like > enumerate experiments that are relevant to a given genome ID; probes that > match a given set of features, retrieval of normalized data for a given > experiment, etc. > > --bob > > On Jul 9, 2013, at 5:45 PM, Thomas Brettin wrote: > > You should talk to Bob if you're wondering about automatically generated > scripts. He, Ross and Terry also wrote many hand crafted scripts as well. > > The poster child I sent you (registry) was for the module Makefile, but it > also includes commands. Additionally, it contains a template file that I > used to auto generate the registry commands (see scripts/script.tt and > scripts/tpage_command), but you should really* talk to Bob* about this as > his auto generate skills are superior to mine. > > Tom > > > > On Jul 9, 2013, at 5:37 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > > Hi Tom, > > Tom you are welcome to play around with my make file. > Yes, this would save me some time, but I have not made scripts and > commands to work in IRIS before. > That is why I was asking about a poster child example. So I really do not > know what is involved. > Currently there are over 30 (and growing) functions that I have made for > expression. > If it is only a wrapper then it should be ok. The automated relationship > and entity functions do not go intp the scripts section. I believe these > get made automatically when I run the CDMI stuff on mmy XML. > The potential issue there is that expression is in a different database. > > -Jason > > > > > On Tue, Jul 9, 2013 at 2:26 PM, Thomas Brettin <brettin(a)mcs.anl.gov>wrote: > >> I haven't enforced a policy on the script requirement yet, although it >> seems having scripts for standard services, particularly those in the CDMI >> space could be required. >> >> I do enforce a policy that all modules must have a COMMANDS file, but it >> is not unheard of that someone will just create an empty file in their >> module. >> >> Jason, if I were to take on cleaning up the make file for the release >> would you then have time to get some commands in place before we release. >> Also, is it possible to automate the generation of the Entity and Relations >> type of commands? I'm sure Bob didn't hand write all those. >> >> Tom >> >> >> On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: >> >> For the initial release we should have command line and iris I would >> think in addition to whatever we have in the API? >> >> We have several others working on pipelines for RNAseq analysis and I >> think we need to harmonize these before too long. >> >> On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >> >> Rick asked for the commands. >> >> >> On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> >> I do not have scripts for iris. This is something that we can >> potentially add later, but not for the initial release. >> >> -Jason >> >> >> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> >>> Hi Tom, >>> >>> I do not pretend to be a make expert. This project is the first time >>> that I have used it. Do you have an example of what you like me to change >>> it to? >>> >>> Thanks, >>> Jason >>> >>> >>> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com>wrote: >>> >>>> Hi Jason, >>>> >>>> I was reviewing your module makefile, and I'd like to help you make >>>> some changes that will rely more on templates. Since your code is all perl, >>>> this should be pretty straight forward. >>>> >>>> As an example, you have in your make file: >>>> >>>> deploy-service-scripts: >>>> # First create the start script (should be a better way to do >>>> this…) >>>> … >>>> >>>> Usually, we use tpage as a templating tool and use start_service.ttand >>>> stop_service.tt templates that get filled in at make time. >>>> >>>> Also, I don't see a scripts directory. Do you have scripts that will be >>>> executable from IRIS? >>>> >>>> Thanks, >>>> Tom >>>> >>>> >>>> >>>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> >>>> Works for me. >>>> >>>> -Jason >>>> >>>> >>>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com>wrote: >>>> >>>>> I'd like to schedule this for a minor release and not wait until the >>>>> Aug build. We'll do at least one minor release this month. >>>>> >>>>> >>>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>> >>>>> > Hi Tom and Paramvir, >>>>> > >>>>> > I think I have an official candidate for release for expression. >>>>> > >>>>> > I have a bunch of stuff but includes the following: >>>>> > spec file >>>>> > impl code with 30 complete functions that have been tested (note 2 >>>>> are currently incomplete functions - related to GEO parsing). >>>>> > automated testing for all completed functions >>>>> > readme >>>>> > releasenotes >>>>> > exchangeformat specs >>>>> > uploader >>>>> > preprocessors >>>>> > initial data loaded into expression on DB1 >>>>> > automated doc for Impl code >>>>> > documentation for exchange format (file format for uploads). >>>>> > >>>>> > I think I have everything except scripts and commands to be run from >>>>> the command line and iris. >>>>> > >>>>> > All of the code is committed to git. >>>>> > >>>>> > -Jason >>>>> >>>>> >>>> >>>> >>> >> _______________________________________________ >> Release-team mailing list >> Release-team(a)lists.kbase.us >> https://lists.kbase.us/mailman/listinfo/release-team >> >> >> >> >> > > >

1 0

Re: [Release-team] Expression Candidate for release
by Robert Olson 10 Jul '13

10 Jul '13

If the expression stuff is actually integrated into the central store data model, it'll have the get_entity/get_rel/etc scripts created as a matter of course. If it's being maintained as a separate database, those will need to be separately generated (compile_dbd_to_typespec & friends). It seems we should have some command line scripts to do things like enumerate experiments that are relevant to a given genome ID; probes that match a given set of features, retrieval of normalized data for a given experiment, etc. --bob On Jul 9, 2013, at 5:45 PM, Thomas Brettin wrote: > You should talk to Bob if you're wondering about automatically generated scripts. He, Ross and Terry also wrote many hand crafted scripts as well. > > The poster child I sent you (registry) was for the module Makefile, but it also includes commands. Additionally, it contains a template file that I used to auto generate the registry commands (see scripts/script.tt and scripts/tpage_command), but you should really talk to Bob about this as his auto generate skills are superior to mine. > > Tom > > > > On Jul 9, 2013, at 5:37 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > >> Hi Tom, >> >> Tom you are welcome to play around with my make file. >> Yes, this would save me some time, but I have not made scripts and commands to work in IRIS before. >> That is why I was asking about a poster child example. So I really do not know what is involved. >> Currently there are over 30 (and growing) functions that I have made for expression. >> If it is only a wrapper then it should be ok. The automated relationship and entity functions do not go intp the scripts section. I believe these get made automatically when I run the CDMI stuff on mmy XML. >> The potential issue there is that expression is in a different database. >> >> -Jason >> >> >> >> >> On Tue, Jul 9, 2013 at 2:26 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >> I haven't enforced a policy on the script requirement yet, although it seems having scripts for standard services, particularly those in the CDMI space could be required. >> >> I do enforce a policy that all modules must have a COMMANDS file, but it is not unheard of that someone will just create an empty file in their module. >> >> Jason, if I were to take on cleaning up the make file for the release would you then have time to get some commands in place before we release. Also, is it possible to automate the generation of the Entity and Relations type of commands? I'm sure Bob didn't hand write all those. >> >> Tom >> >> >> On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: >> >>> For the initial release we should have command line and iris I would think in addition to whatever we have in the API? >>> >>> We have several others working on pipelines for RNAseq analysis and I think we need to harmonize these before too long. >>> >>> On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >>> >>>> Rick asked for the commands. >>>> >>>> >>>> On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> >>>>> I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. >>>>> >>>>> -Jason >>>>> >>>>> >>>>> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>> Hi Tom, >>>>> >>>>> I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? >>>>> >>>>> Thanks, >>>>> Jason >>>>> >>>>> >>>>> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>> Hi Jason, >>>>> >>>>> I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. >>>>> >>>>> As an example, you have in your make file: >>>>> >>>>> deploy-service-scripts: >>>>> # First create the start script (should be a better way to do this…) >>>>> … >>>>> >>>>> Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. >>>>> >>>>> Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? >>>>> >>>>> Thanks, >>>>> Tom >>>>> >>>>> >>>>> >>>>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>> >>>>>> Works for me. >>>>>> >>>>>> -Jason >>>>>> >>>>>> >>>>>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>>> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >>>>>> >>>>>> >>>>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>>> >>>>>> > Hi Tom and Paramvir, >>>>>> > >>>>>> > I think I have an official candidate for release for expression. >>>>>> > >>>>>> > I have a bunch of stuff but includes the following: >>>>>> > spec file >>>>>> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >>>>>> > automated testing for all completed functions >>>>>> > readme >>>>>> > releasenotes >>>>>> > exchangeformat specs >>>>>> > uploader >>>>>> > preprocessors >>>>>> > initial data loaded into expression on DB1 >>>>>> > automated doc for Impl code >>>>>> > documentation for exchange format (file format for uploads). >>>>>> > >>>>>> > I think I have everything except scripts and commands to be run from the command line and iris. >>>>>> > >>>>>> > All of the code is committed to git. >>>>>> > >>>>>> > -Jason >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Release-team mailing list >>>>> Release-team(a)lists.kbase.us >>>>> https://lists.kbase.us/mailman/listinfo/release-team >>>> >>> >> >> >

1 0

Re: [Release-team] Expression Candidate for release
by Thomas Brettin 09 Jul '13

09 Jul '13

You should talk to Bob if you're wondering about automatically generated scripts. He, Ross and Terry also wrote many hand crafted scripts as well. The poster child I sent you (registry) was for the module Makefile, but it also includes commands. Additionally, it contains a template file that I used to auto generate the registry commands (see scripts/script.tt and scripts/tpage_command), but you should really talk to Bob about this as his auto generate skills are superior to mine. Tom On Jul 9, 2013, at 5:37 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > Hi Tom, > > Tom you are welcome to play around with my make file. > Yes, this would save me some time, but I have not made scripts and commands to work in IRIS before. > That is why I was asking about a poster child example. So I really do not know what is involved. > Currently there are over 30 (and growing) functions that I have made for expression. > If it is only a wrapper then it should be ok. The automated relationship and entity functions do not go intp the scripts section. I believe these get made automatically when I run the CDMI stuff on mmy XML. > The potential issue there is that expression is in a different database. > > -Jason > > > > > On Tue, Jul 9, 2013 at 2:26 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > I haven't enforced a policy on the script requirement yet, although it seems having scripts for standard services, particularly those in the CDMI space could be required. > > I do enforce a policy that all modules must have a COMMANDS file, but it is not unheard of that someone will just create an empty file in their module. > > Jason, if I were to take on cleaning up the make file for the release would you then have time to get some commands in place before we release. Also, is it possible to automate the generation of the Entity and Relations type of commands? I'm sure Bob didn't hand write all those. > > Tom > > > On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: > >> For the initial release we should have command line and iris I would think in addition to whatever we have in the API? >> >> We have several others working on pipelines for RNAseq analysis and I think we need to harmonize these before too long. >> >> On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >> >>> Rick asked for the commands. >>> >>> >>> On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> >>>> I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. >>>> >>>> -Jason >>>> >>>> >>>> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> Hi Tom, >>>> >>>> I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? >>>> >>>> Thanks, >>>> Jason >>>> >>>> >>>> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>> Hi Jason, >>>> >>>> I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. >>>> >>>> As an example, you have in your make file: >>>> >>>> deploy-service-scripts: >>>> # First create the start script (should be a better way to do this…) >>>> … >>>> >>>> Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. >>>> >>>> Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? >>>> >>>> Thanks, >>>> Tom >>>> >>>> >>>> >>>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> >>>>> Works for me. >>>>> >>>>> -Jason >>>>> >>>>> >>>>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >>>>> >>>>> >>>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>> >>>>> > Hi Tom and Paramvir, >>>>> > >>>>> > I think I have an official candidate for release for expression. >>>>> > >>>>> > I have a bunch of stuff but includes the following: >>>>> > spec file >>>>> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >>>>> > automated testing for all completed functions >>>>> > readme >>>>> > releasenotes >>>>> > exchangeformat specs >>>>> > uploader >>>>> > preprocessors >>>>> > initial data loaded into expression on DB1 >>>>> > automated doc for Impl code >>>>> > documentation for exchange format (file format for uploads). >>>>> > >>>>> > I think I have everything except scripts and commands to be run from the command line and iris. >>>>> > >>>>> > All of the code is committed to git. >>>>> > >>>>> > -Jason >>>>> >>>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Release-team mailing list >>>> Release-team(a)lists.kbase.us >>>> https://lists.kbase.us/mailman/listinfo/release-team >>> >> > >

1 0

Re: [Release-team] Expression Candidate for release
by Jason Baumohl 09 Jul '13

09 Jul '13

Hi Tom, Tom you are welcome to play around with my make file. Yes, this would save me some time, but I have not made scripts and commands to work in IRIS before. That is why I was asking about a poster child example. So I really do not know what is involved. Currently there are over 30 (and growing) functions that I have made for expression. If it is only a wrapper then it should be ok. The automated relationship and entity functions do not go intp the scripts section. I believe these get made automatically when I run the CDMI stuff on mmy XML. The potential issue there is that expression is in a different database. -Jason On Tue, Jul 9, 2013 at 2:26 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > I haven't enforced a policy on the script requirement yet, although it > seems having scripts for standard services, particularly those in the CDMI > space could be required. > > I do enforce a policy that all modules must have a COMMANDS file, but it > is not unheard of that someone will just create an empty file in their > module. > > Jason, if I were to take on cleaning up the make file for the release > would you then have time to get some commands in place before we release. > Also, is it possible to automate the generation of the Entity and Relations > type of commands? I'm sure Bob didn't hand write all those. > > Tom > > > On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: > > For the initial release we should have command line and iris I would think > in addition to whatever we have in the API? > > We have several others working on pipelines for RNAseq analysis and I > think we need to harmonize these before too long. > > On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > > Rick asked for the commands. > > > On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > > I do not have scripts for iris. This is something that we can potentially > add later, but not for the initial release. > > -Jason > > > On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > >> Hi Tom, >> >> I do not pretend to be a make expert. This project is the first time >> that I have used it. Do you have an example of what you like me to change >> it to? >> >> Thanks, >> Jason >> >> >> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com>wrote: >> >>> Hi Jason, >>> >>> I was reviewing your module makefile, and I'd like to help you make some >>> changes that will rely more on templates. Since your code is all perl, this >>> should be pretty straight forward. >>> >>> As an example, you have in your make file: >>> >>> deploy-service-scripts: >>> # First create the start script (should be a better way to do >>> this…) >>> … >>> >>> Usually, we use tpage as a templating tool and use start_service.tt and >>> stop_service.tt templates that get filled in at make time. >>> >>> Also, I don't see a scripts directory. Do you have scripts that will be >>> executable from IRIS? >>> >>> Thanks, >>> Tom >>> >>> >>> >>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> >>> Works for me. >>> >>> -Jason >>> >>> >>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com>wrote: >>> >>>> I'd like to schedule this for a minor release and not wait until the >>>> Aug build. We'll do at least one minor release this month. >>>> >>>> >>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> >>>> > Hi Tom and Paramvir, >>>> > >>>> > I think I have an official candidate for release for expression. >>>> > >>>> > I have a bunch of stuff but includes the following: >>>> > spec file >>>> > impl code with 30 complete functions that have been tested (note 2 >>>> are currently incomplete functions - related to GEO parsing). >>>> > automated testing for all completed functions >>>> > readme >>>> > releasenotes >>>> > exchangeformat specs >>>> > uploader >>>> > preprocessors >>>> > initial data loaded into expression on DB1 >>>> > automated doc for Impl code >>>> > documentation for exchange format (file format for uploads). >>>> > >>>> > I think I have everything except scripts and commands to be run from >>>> the command line and iris. >>>> > >>>> > All of the code is committed to git. >>>> > >>>> > -Jason >>>> >>>> >>> >>> >> > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team > > > > >

1 0

Re: [Release-team] Expression Candidate for release
by Rick stevens 09 Jul '13

09 Jul '13

I think it's time we require good command line scripts for everything. I'm not picking on Jason. It's overdue. We need to be able to do scripted end to end testing etc. Sent from my iPad On Jul 9, 2013, at 4:26 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > I haven't enforced a policy on the script requirement yet, although it seems having scripts for standard services, particularly those in the CDMI space could be required. > > I do enforce a policy that all modules must have a COMMANDS file, but it is not unheard of that someone will just create an empty file in their module. > > Jason, if I were to take on cleaning up the make file for the release would you then have time to get some commands in place before we release. Also, is it possible to automate the generation of the Entity and Relations type of commands? I'm sure Bob didn't hand write all those. > > Tom > > > On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: > >> For the initial release we should have command line and iris I would think in addition to whatever we have in the API? >> >> We have several others working on pipelines for RNAseq analysis and I think we need to harmonize these before too long. >> >> On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: >> >>> Rick asked for the commands. >>> >>> >>> On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> >>>> I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. >>>> >>>> -Jason >>>> >>>> >>>> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>> Hi Tom, >>>>> >>>>> I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? >>>>> >>>>> Thanks, >>>>> Jason >>>>> >>>>> >>>>> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>>> Hi Jason, >>>>>> >>>>>> I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. >>>>>> >>>>>> As an example, you have in your make file: >>>>>> >>>>>> deploy-service-scripts: >>>>>> # First create the start script (should be a better way to do this…) >>>>>> … >>>>>> >>>>>> Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. >>>>>> >>>>>> Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? >>>>>> >>>>>> Thanks, >>>>>> Tom >>>>>> >>>>>> >>>>>> >>>>>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>>> >>>>>>> Works for me. >>>>>>> >>>>>>> -Jason >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>>>>>> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >>>>>>>> >>>>>>>> >>>>>>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>>>>>> >>>>>>>> > Hi Tom and Paramvir, >>>>>>>> > >>>>>>>> > I think I have an official candidate for release for expression. >>>>>>>> > >>>>>>>> > I have a bunch of stuff but includes the following: >>>>>>>> > spec file >>>>>>>> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >>>>>>>> > automated testing for all completed functions >>>>>>>> > readme >>>>>>>> > releasenotes >>>>>>>> > exchangeformat specs >>>>>>>> > uploader >>>>>>>> > preprocessors >>>>>>>> > initial data loaded into expression on DB1 >>>>>>>> > automated doc for Impl code >>>>>>>> > documentation for exchange format (file format for uploads). >>>>>>>> > >>>>>>>> > I think I have everything except scripts and commands to be run from the command line and iris. >>>>>>>> > >>>>>>>> > All of the code is committed to git. >>>>>>>> > >>>>>>>> > -Jason >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Release-team mailing list >>>> Release-team(a)lists.kbase.us >>>> https://lists.kbase.us/mailman/listinfo/release-team >>> >> >

1 0

Re: [Release-team] Expression Candidate for release
by Thomas Brettin 09 Jul '13

09 Jul '13

I haven't enforced a policy on the script requirement yet, although it seems having scripts for standard services, particularly those in the CDMI space could be required. I do enforce a policy that all modules must have a COMMANDS file, but it is not unheard of that someone will just create an empty file in their module. Jason, if I were to take on cleaning up the make file for the release would you then have time to get some commands in place before we release. Also, is it possible to automate the generation of the Entity and Relations type of commands? I'm sure Bob didn't hand write all those. Tom On Jul 9, 2013, at 3:17 PM, Rick Stevens <stevens(a)anl.gov> wrote: > For the initial release we should have command line and iris I would think in addition to whatever we have in the API? > > We have several others working on pipelines for RNAseq analysis and I think we need to harmonize these before too long. > > On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > >> Rick asked for the commands. >> >> >> On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> >>> I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. >>> >>> -Jason >>> >>> >>> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> Hi Tom, >>> >>> I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? >>> >>> Thanks, >>> Jason >>> >>> >>> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: >>> Hi Jason, >>> >>> I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. >>> >>> As an example, you have in your make file: >>> >>> deploy-service-scripts: >>> # First create the start script (should be a better way to do this…) >>> … >>> >>> Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. >>> >>> Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? >>> >>> Thanks, >>> Tom >>> >>> >>> >>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> >>>> Works for me. >>>> >>>> -Jason >>>> >>>> >>>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >>>> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >>>> >>>> >>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> >>>> > Hi Tom and Paramvir, >>>> > >>>> > I think I have an official candidate for release for expression. >>>> > >>>> > I have a bunch of stuff but includes the following: >>>> > spec file >>>> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >>>> > automated testing for all completed functions >>>> > readme >>>> > releasenotes >>>> > exchangeformat specs >>>> > uploader >>>> > preprocessors >>>> > initial data loaded into expression on DB1 >>>> > automated doc for Impl code >>>> > documentation for exchange format (file format for uploads). >>>> > >>>> > I think I have everything except scripts and commands to be run from the command line and iris. >>>> > >>>> > All of the code is committed to git. >>>> > >>>> > -Jason >>>> >>>> >>> >>> >>> >>> _______________________________________________ >>> Release-team mailing list >>> Release-team(a)lists.kbase.us >>> https://lists.kbase.us/mailman/listinfo/release-team >> >

1 0

Re: [Release-team] Expression Candidate for release
by Rick Stevens 09 Jul '13

09 Jul '13

For the initial release we should have command line and iris I would think in addition to whatever we have in the API? We have several others working on pipelines for RNAseq analysis and I think we need to harmonize these before too long. On Jul 9, 2013, at 2:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > Rick asked for the commands. > > > On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > >> I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. >> >> -Jason >> >> >> On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> Hi Tom, >> >> I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? >> >> Thanks, >> Jason >> >> >> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: >> Hi Jason, >> >> I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. >> >> As an example, you have in your make file: >> >> deploy-service-scripts: >> # First create the start script (should be a better way to do this…) >> … >> >> Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. >> >> Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? >> >> Thanks, >> Tom >> >> >> >> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> >>> Works for me. >>> >>> -Jason >>> >>> >>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >>> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >>> >>> >>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> >>> > Hi Tom and Paramvir, >>> > >>> > I think I have an official candidate for release for expression. >>> > >>> > I have a bunch of stuff but includes the following: >>> > spec file >>> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >>> > automated testing for all completed functions >>> > readme >>> > releasenotes >>> > exchangeformat specs >>> > uploader >>> > preprocessors >>> > initial data loaded into expression on DB1 >>> > automated doc for Impl code >>> > documentation for exchange format (file format for uploads). >>> > >>> > I think I have everything except scripts and commands to be run from the command line and iris. >>> > >>> > All of the code is committed to git. >>> > >>> > -Jason >>> >>> >> >> >> >> _______________________________________________ >> Release-team mailing list >> Release-team(a)lists.kbase.us >> https://lists.kbase.us/mailman/listinfo/release-team >

1 0

Re: [Release-team] Expression Candidate for release
by Jason Baumohl 09 Jul '13

09 Jul '13

Hi Tom, After getting the GEO metadata parser and make file done, I would then like to get Sunita up to speed so she can start working on getting the plants data loaded. Then I can shift gears and look into doing this. Is there a poster child example service you would recommend me looking at the scripts. Thanks, Jason On Tue, Jul 9, 2013 at 12:47 PM, Thomas Brettin <brettin(a)mcs.anl.gov> wrote: > Rick asked for the commands. > > > On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > > I do not have scripts for iris. This is something that we can potentially > add later, but not for the initial release. > > -Jason > > > On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > >> Hi Tom, >> >> I do not pretend to be a make expert. This project is the first time >> that I have used it. Do you have an example of what you like me to change >> it to? >> >> Thanks, >> Jason >> >> >> On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com>wrote: >> >>> Hi Jason, >>> >>> I was reviewing your module makefile, and I'd like to help you make some >>> changes that will rely more on templates. Since your code is all perl, this >>> should be pretty straight forward. >>> >>> As an example, you have in your make file: >>> >>> deploy-service-scripts: >>> # First create the start script (should be a better way to do >>> this…) >>> … >>> >>> Usually, we use tpage as a templating tool and use start_service.tt and >>> stop_service.tt templates that get filled in at make time. >>> >>> Also, I don't see a scripts directory. Do you have scripts that will be >>> executable from IRIS? >>> >>> Thanks, >>> Tom >>> >>> >>> >>> On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>> >>> Works for me. >>> >>> -Jason >>> >>> >>> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com>wrote: >>> >>>> I'd like to schedule this for a minor release and not wait until the >>>> Aug build. We'll do at least one minor release this month. >>>> >>>> >>>> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >>>> >>>> > Hi Tom and Paramvir, >>>> > >>>> > I think I have an official candidate for release for expression. >>>> > >>>> > I have a bunch of stuff but includes the following: >>>> > spec file >>>> > impl code with 30 complete functions that have been tested (note 2 >>>> are currently incomplete functions - related to GEO parsing). >>>> > automated testing for all completed functions >>>> > readme >>>> > releasenotes >>>> > exchangeformat specs >>>> > uploader >>>> > preprocessors >>>> > initial data loaded into expression on DB1 >>>> > automated doc for Impl code >>>> > documentation for exchange format (file format for uploads). >>>> > >>>> > I think I have everything except scripts and commands to be run from >>>> the command line and iris. >>>> > >>>> > All of the code is committed to git. >>>> > >>>> > -Jason >>>> >>>> >>> >>> >> > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team > > >

1 0

Re: [Release-team] Expression Candidate for release
by Thomas Brettin 09 Jul '13

09 Jul '13

Rick asked for the commands. On Jul 9, 2013, at 1:51 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > I do not have scripts for iris. This is something that we can potentially add later, but not for the initial release. > > -Jason > > > On Tue, Jul 9, 2013 at 11:42 AM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > Hi Tom, > > I do not pretend to be a make expert. This project is the first time that I have used it. Do you have an example of what you like me to change it to? > > Thanks, > Jason > > > On Tue, Jul 9, 2013 at 6:34 AM, Thomas Brettin <brettint(a)gmail.com> wrote: > Hi Jason, > > I was reviewing your module makefile, and I'd like to help you make some changes that will rely more on templates. Since your code is all perl, this should be pretty straight forward. > > As an example, you have in your make file: > > deploy-service-scripts: > # First create the start script (should be a better way to do this…) > … > > Usually, we use tpage as a templating tool and use start_service.tt and stop_service.tt templates that get filled in at make time. > > Also, I don't see a scripts directory. Do you have scripts that will be executable from IRIS? > > Thanks, > Tom > > > > On Jul 8, 2013, at 7:49 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: > >> Works for me. >> >> -Jason >> >> >> On Mon, Jul 8, 2013 at 5:40 PM, Thomas Brettin <brettint(a)gmail.com> wrote: >> I'd like to schedule this for a minor release and not wait until the Aug build. We'll do at least one minor release this month. >> >> >> On Jul 1, 2013, at 5:06 PM, Jason Baumohl <jkbaumohl(a)lbl.gov> wrote: >> >> > Hi Tom and Paramvir, >> > >> > I think I have an official candidate for release for expression. >> > >> > I have a bunch of stuff but includes the following: >> > spec file >> > impl code with 30 complete functions that have been tested (note 2 are currently incomplete functions - related to GEO parsing). >> > automated testing for all completed functions >> > readme >> > releasenotes >> > exchangeformat specs >> > uploader >> > preprocessors >> > initial data loaded into expression on DB1 >> > automated doc for Impl code >> > documentation for exchange format (file format for uploads). >> > >> > I think I have everything except scripts and commands to be run from the command line and iris. >> > >> > All of the code is committed to git. >> > >> > -Jason >> >> > > > > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team

1 0