February 2014 - Release-team - mailman3.cels.anl.gov

Release team sync
by Gordon, India S. 04 Feb '14

04 Feb '14

When: Tuesday, February 04, 2014 3:00 PM-3:30 PM. (UTC-06:00) Central Time (US & Canada) Where: 888-850-4523, passcode: 416791, leader passcode (Tom only) 4167913 *~*~*~*~*~*~*~*~*~*

1 0

FW: KBaseExpression GIT PUSHED
by Land, Miriam L. 04 Feb '14

04 Feb '14

KBaseExpression is now ready to be deployed and tested. I'll be watching the services spreadsheet to see if there is anything I need to do. I'm expecting the last of Pavel's modules to be checked into git by 8am Pacific time. We should be ready to deploy/test and move to prod after that. I'll try to get out a status report in about an hour. Miriam From: Jason Baumohl <jkbaumohl(a)lbl.gov<mailto:[email protected]>> Date: Tuesday, February 4, 2014 1:26 AM To: Pavel Novichkov <PSNovichkov(a)lbl.gov<mailto:[email protected]>>, Miriam Land <landml(a)ornl.gov<mailto:[email protected]>> Cc: Shinjae Yoo <sjyoo(a)bnl.gov<mailto:[email protected]>>, "Thomas S. Brettin" <brettin(a)cels.anl.gov<mailto:[email protected]>> Subject: Re: GIT PUSHED Hi Miriam, KBaseExpression is now ready to be tested. It is currently pointing to the dev database. So later we will need to change the deploy.cfg over. However in the meantime it can be tested vs the dev DB. Thanks, Jason On Mon, Feb 3, 2014 at 10:26 PM, Pavel Novichkov <psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: Jason, Please send a note to Miriam. Thanks, P. On Feb 3, 2014, at 10:21 PM, Jason Baumohl <jkbaumohl(a)lbl.gov<mailto:[email protected]>> wrote: OK changes made, tested and committed. -Jason On Mon, Feb 3, 2014 at 9:45 PM, Pavel Novichkov <psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: host = db4.chicago.kbase.us<http://db4.chicago.kbase.us/> dbname = kbase_sapling_v3_dev user = kbase_sapselect pass = oiwn22&dmwWEe Please try and be sure that you can see expression data, thanks. P. On Feb 3, 2014, at 9:39 PM, Jason Baumohl <jkbaumohl(a)lbl.gov<mailto:[email protected]>> wrote: Hi Pavel, Please clarify. Is there a read only account? If so what is the user name and pwd? I can put it into the deploy.cfg -Jason On Mon, Feb 3, 2014 at 9:25 PM, Pavel Novichkov <psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: Shinjae, We can put MySQL credentials into deploy.cfg files… it can be pointed to kbase_sapling_v3_dev on db4.chicago.kbase.us<http://db4.chicago.kbase.us/> for testing… and then deployment team will modify it and point to production CS…. What do you think? Can you do it? Jason, I think you can do the same… (actually please do it..). Let us use kbase_sapling_v3_dev on db4.chicago.kbase.us<http://db4.chicago.kbase.us/> and "read-only" user name and pwd by default... P. On Feb 3, 2014, at 7:33 PM, "Yoo, Shinjae" <sjyoo(a)bnl.gov<mailto:[email protected]>> wrote: Pavel, I’m playing with dataset and one thing is odd: we lost strength >From original dataset you loaded: {"name":"<->","dataset_id":"kb|g.3899.nds.12","properties":{},"strength":1.04554164409637,"node_id2":"kb|netnode.7691","directed":"false","user_annotations":{},"id":"kb|g.3899.nds.12.edge.12","confidence":0,"node_id1":"kb|netnode.10084”} >From networks api output: {"id":"kb|netedge.2474","name":"is interact with","node_id1":"kb|netnode.1","node_id2":"kb|netnode.1238","directed":"false","strength":0.0,"confidence":0.0,"dataset_id":"kb|g.3899.nds.12","properties":{},"user_annotations":{}} >From database table: mysql> select * from AssociationFeature where to_link = 'kb|g.3899.CDS.68927' limit 10; +------------------------------+---------------------+------+---------------+----------+ | from_link | to_link | rank | stoichiometry | strength | +------------------------------+---------------------+------+---------------+----------+ | kb|g.3899.nds.12.edge.93313 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.99086 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.100667 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.115777 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.121994 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.126242 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.127387 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.149415 | kb|g.3899.CDS.68927 | 2 | 0 | 0 | | kb|g.3899.nds.12.edge.163204 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | | kb|g.3899.nds.12.edge.164735 | kb|g.3899.CDS.68927 | 1 | 0 | 0 | +------------------------------+---------------------+------+---------------+----------+ 10 rows in set (0.00 sec) mysql> select * from Association where id = 'kb|g.3899.nds.12.edge.93313'; +-----------------------------+------------+-------------+-------------+------+-----+ | id | confidence | directional | description | name | url | +-----------------------------+------------+-------------+-------------+------+-----+ | kb|g.3899.nds.12.edge.93313 | 0 | 0 | | <-> | | +-----------------------------+------------+-------------+-------------+------+-----+ 1 row in set (0.00 sec) It seems we lost strength… Thanks, -SJ On Feb 3, 2014, at 9:56 PM, Shinjae Yoo <sjyoo(a)bnl.gov<mailto:[email protected]>> wrote: Yup, mysql> use kbase_sapling_v3_dev; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select count(*) from AssociationDataset; +----------+ | count(*) | +----------+ | 265 | +----------+ 1 row in set (0.01 sec) I could see that now… I was using v3_empty instead of v3_dev… I’m generating test cases… Thanks, -SJ On Feb 3, 2014, at 9:57 PM, Pavel Novichkov <psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: I just reconnected to db4.chicago.kbase.us<http://db4.chicago.kbase.us/> and kbase_sapling_v3_dev from your instance and see everything: mysql> select count(*) from AssociationDataset; +----------+ | count(*) | +----------+ | 265 | +----------+ 1 row in set (0.00 sec) Can you reproduce it? Unfortunately I thought that the data was not loaded correctly and loaded one JM's PPI dataset: kbase_sapling_v3_dev.AssociationDataset: Records: 1 Deleted: 0 Skipped: 1 Warnings: 1 kbase_sapling_v3_dev.IsDatasetFor: Records: 1 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.IsGroupingOf: Records: 110 Deleted: 0 Skipped: 110 Warnings: 110 kbase_sapling_v3_dev.Association: Records: 110 Deleted: 0 Skipped: 110 Warnings: 110 kbase_sapling_v3_dev.AssociationFeature: Records: 220 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.AssociationDetectionType: Records: 1 Deleted: 0 Skipped: 1 Warnings: 1 kbase_sapling_v3_dev.DetectedWithMethod: Records: 110 Deleted: 0 Skipped: 110 Warnings: 110 kbase_sapling_v3_dev.Publication: Records: 1 Deleted: 0 Skipped: 1 Warnings: 1 kbase_sapling_v3_dev.PublishedAssociation: Records: 110 Deleted: 0 Skipped: 110 Warnings: 110 kbase_sapling_v3_dev.Source: Records: 1 Deleted: 0 Skipped: 1 Warnings: 1 kbase_sapling_v3_dev.IsSourceForAssociationDataset: Records: 1 Deleted: 0 Skipped: 1 Warnings: 1 Most records were fortunately skipped, but for tables they are now duplicated…. So, we might have wrong data for: kb|ppid.0 P. On Feb 3, 2014, at 6:43 PM, "Yoo, Shinjae" <sjyoo(a)bnl.gov<mailto:[email protected]>> wrote: Ah you directly touched base_sapling_v3_dev… I’ll change db from v3_empty to v3_dev… Thanks, -SJ On Feb 3, 2014, at 9:36 PM, Pavel Novichkov <psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: I do not undertand that…. Yes, I see that there are only 4 datasets… but here is the statistics of loading: Mon Feb 3 22:26:32 UTC 2014 kbase_sapling_v3_dev.AssociationDetectionType: Records: 0 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.DetectedWithMethod: Records: 0 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.Association: Records: 51099805 Deleted: 0 Skipped: 3619 Warnings: 3619 kbase_sapling_v3_dev.AssociationFeature: Records: 102313637 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.IsGroupingOf: Records: 51099805 Deleted: 0 Skipped: 3619 Warnings: 3619 kbase_sapling_v3_dev.PublishedAssociation: Records: 0 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.Publication: Records: 0 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.AssociationDataset: Records: 256 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.IsDatasetFor: Records: 258 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.IsSourceForAssociationDataset: Records: 256 Deleted: 0 Skipped: 0 Warnings: 0 kbase_sapling_v3_dev.Source: Records: 256 Deleted: 0 Skipped: 256 Warnings: 256 Tue Feb 4 00:41:34 UTC 2014 256 records should be loaded… not warnings, no skipping…. P. On Feb 3, 2014, at 6:21 PM, "Yoo, Shinjae" <sjyoo(a)bnl.gov<mailto:[email protected]>> wrote: Pavel, mysql> select count(*) from AssociationDataset; +----------+ | count(*) | +----------+ | 4 | +----------+ 1 row in set (0.00 sec) mysql> select * from AssociationDataset limit 10; ERROR 2006 (HY000): MySQL server has gone away No connection. Trying to reconnect... Connection id: 2697 Current database: kbase_sapling_v3_empty Am I using the same database with you? Thanks, -SJ On Feb 3, 2014, at 9:11 PM, Shinjae Yoo <sjyoo(a)bnl.gov<mailto:[email protected]>> wrote: Great… Thanks… I’ll test it… -SJ On Feb 3, 2014, at 9:00 PM, Pavel Novichkov <psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: Shinjae, All data is loaded to kbase_sapling_v3_dev. P. On Feb 3, 2014, at 1:04 PM, "Yoo, Shinjae" <sjyoo(a)bnl.gov<mailto:[email protected]>> wrote: Pavel, I just pushed… Now I’m leaving for home… if you have burning questions, just call me `412-467-6860<tel:412-467-6860>’… I should be able to answer your call… Thanks, -SJ

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Brettin, Thomas S. 03 Feb '14

03 Feb '14

I want to see the make deploy target depend on a subtarget that registers the typespec. When make deploy is run in dev, it registers the spec in dev When make deploy is run in test, it registers the spec in test and same in prod. I'm working on a prototype of this that I'll write up as a developer guide and bake into the template makefile. It should apply to KBase types, not community types. Deployment to production should be fast, not a cumbersome process On Feb 3, 2014, at 2:24 PM, Gavin A Price <gavinaprice(a)gmail.com<mailto:[email protected]>> wrote: Ok, so if you want to exactly duplicate the prod type db in the test server, you have a couple options 1) Blow away the current type db and the workspace db, starting from scratch, and dump the type db from prod and restore it to test. Initialize the workspace db as normal and point it as the restored type db. 2) Go through the standard process of users requesting module ownership, granting them ownership via the admin function, and then have them upload their types as they did in the prod db. Shortcuts are pretty much what I outlined before: 1) Grab all the most recent modules from the workspace service, a test team member requests ownership of all modules, approves via the admin function, and loads all the modules. This will not load older versions of modules, which may or may not be ok. 2) Give all the users admin access so they can approve their own modules. See https://atlassian.kbase.us/browse/WOR-80 for a potential long term solution. Long term, if you guys could outline how you'd like testing of types to work we can try to add the necessary functionality to the service. -g On 2/3/2014 9:17 AM, Gavin Price wrote: No, that's what I was saying in my response to Tom. You'd have to get the types manually, one by one, and have the owners go through the approval process on the test server and redo the uploads if you want to duplicate the prod db. There are short cuts possible, though. I need to deal with some moving stuff now so I'll outline those when I get in. -g Sent from my Cray XT30 On Feb 3, 2014 9:10 AM, "Shane Canon" <scanon(a)lbl.gov<mailto:[email protected]>> wrote: I think a dump from prod of the types and load to test makes sense. This allows us to at least be somewhat in sync with the production environment. I don't think it is feasible to have all the data, but it is reasonable to have the types and maybe some sample data. Gavin: Do APIs and commands presently exist to do this? --Shane On Feb 2, 2014, at 7:56 PM, Gavin Price <gaprice(a)lbl.gov<mailto:[email protected]>> wrote: Sent from my Cray XT30 On Feb 2, 2014 5:07 PM, "Brettin, Thomas S." <brettin(a)cels.anl.gov<mailto:[email protected]>> wrote: > > -------- > WARNING: At least one of the links in the message below goes to an IP address (e.g.10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > -------- > > We need a two step process. Dump all types in prod. Load onto test. That's not going to happen in the next couple weeks. > Seems backwards. It'll do for now. How we manage types in kbase needs an overall rethink, as we discussed. -g > > Sent from my Verizon Wireless 4G LTE DROID > > > Gavin Price <gaprice(a)lbl.gov<mailto:[email protected]>> wrote: > > You don't need to review them, and the mechanics of approving a type (or > a set of types) takes a couple seconds. I would just say that if a type > is available in the prod server it should be available in the test > server, and the prod server can act as the list of approved types. All > the test team needs to do is see if the requester owns the type in the > prod server and if so, approve it on the test server. This could > probably be scripted easily. > > > Why is type registration a risky operation? > Giving all users admin privileges on the test workspace service is what > I was saying seems dangerous. > > -g > > On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > > -------- > > WARNING: At least one of the links in the message below goes to an IP address (e.g. 10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > > -------- > > > > I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. > > > > Why is type registration a risky operation? > > > > --Dan > > > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov<mailto:[email protected]>> wrote: > >> > >> Firstly, you may want to check out > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > >> > >> These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. > >> > >> There are a few options that are possible: > >> > >> 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. > >> > >> 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it > >> > >> 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. > >> > >> -g > >> > >> > >>> On 2/2/2014 8:57 AM, Shane Canon wrote: > >>> Alexey, > >>> > >>> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has > >>> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. > >>> > >>> --Shane > >>> > >>> > >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov<mailto:[email protected]>> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: > >>>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). > >>>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. > >>>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. > >>>> 4. Module's owner releases module for general use of its type (Workspace.release_module). > >>>> > >>>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. > >>>> > >>>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. > >>>> > >>>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? > >>>> > >>>> Best, > >>>> Alexey > >>>> > >>>> > >>>> > >>>> > >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: > >>>>> I'm adding the release-team to the thread so they are in the loop. > >>>>> > >>>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production > >>>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in > >>>>> the workspace, it should populate it as part of the test. > >>>>> > >>>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? > >>>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded > >>>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded > >>>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. > >>>>> > >>>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but > >>>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in > >>>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it > >>>>> is necessary (i.e. a major change in the type compiler or an API change). > >>>>> > >>>>> --Shane > >>>>> > >>>>> > >>>>> > >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov<mailto:[email protected]>> wrote: > >>>>>> > >>>>>> Hi Shane, > >>>>>> > >>>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? > >>>>>> > >>>>>> Thank you, > >>>>>> Alexey > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: > >>>>>>> Pavel, > >>>>>>> > >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. > >>>>>>> > >>>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't > >>>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. > >>>>>>> > >>>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing > >>>>>>> against a workspace service that isn't the production one. So it is probably missing some > >>>>>>> objects that are used in the test. I would like to have a test I can run in the test environment > >>>>>>> just for completeness. But I think we are close. > >>>>>>> > >>>>>>> --Shane > >>>>>>> > >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: > >>>>>>>> > >>>>>>>> Hi Shane, > >>>>>>>> > >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> P. > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov<mailto:[email protected]>> wrote: > >>>>>>>>> > >>>>>>>>> Pavel, > >>>>>>>>> > >>>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for > >>>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. > >>>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references > >>>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. > >>>>>>>>> So that needs to be fixed. > >>>>>>>>> > >>>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. > >>>>>>>>> > >>>>>>>>> For now, I will work on bambi and inferelator. > >>>>>>>>> > >>>>>>>>> --Shane > >>>>>>>>> > >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test > >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' > >>>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' > >>>>>>>>> # run each test > >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ > >>>>>>>>> if [ $? -ne 0 ] ; then \ > >>>>>>>>> exit 1 ; \ > >>>>>>>>> fi \ > >>>>>>>>> > >>>>>>>>> 1..24 > >>>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws > >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws > >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 32. > >>>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat > su > >> n.reflect.Deleg > >>>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteA > da > >> pter.service(Coy > >>>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.griz > zl > >> y.SelectionKeyCo > >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} > >>>>>>>>> > >>>>>>>>> not ok 3 - MEME run successfuly > >>>>>>>>> # Failed test 'MEME run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 35. > >>>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws > >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws > >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 6 - MEME run successfuly > >>>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws > >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 50. > >>>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccesso > rI > >> mpl.invoke(Nati > >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. > ja > >> va:331)\n\tat or > >>>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall > (P > >> rotocolChainCont > >>>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} > >>>>>>>>> > >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly > >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 53. > >>>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws > >>>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 59. > >>>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:77 > 0) > >> \n\tat org.apac > >>>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask > .d > >> oProcess(Process > >>>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"6 > 56 > >> 565433923738"} > >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly > >>>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws > >>>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws > >>>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 69. > >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> not ok 15 - TOMTOM run successfuly > >>>>>>>>> # Failed test 'TOMTOM run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 72. > >>>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws > >>>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws > >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 18 - TOMTOM run successfuly > >>>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws > >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws > >>>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 87. > >>>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl. > in > >> voke(NativeMeth > >>>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 > 31 > >> )\n\tat org.apac > >>>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Proto > co > >> lChainContextTas > >>>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} > >>>>>>>>> > >>>>>>>>> not ok 21 - MAST run successfuly > >>>>>>>>> # Failed test 'MAST run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 90. > >>>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws > >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws > >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 24 - MAST run successfuly > >>>>>>>>> # Looks like you failed 9 tests of 24. > >>>>>>>>> make: *** [test-scripts] Error 1 > >> _______________________________________________ > >> Release-team mailing list > >> Release-team(a)lists.kbase.us<mailto:[email protected]> > >> https://lists.kbase.us/mailman/listinfo/release-team > > > > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us<mailto:[email protected]> > https://lists.kbase.us/mailman/listinfo/release-team _______________________________________________ Release-team mailing list Release-team(a)lists.kbase.us<mailto:[email protected]> https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Gavin A Price 03 Feb '14

03 Feb '14

Ok, so if you want to exactly duplicate the prod type db in the test server, you have a couple options 1) Blow away the current type db and the workspace db, starting from scratch, and dump the type db from prod and restore it to test. Initialize the workspace db as normal and point it as the restored type db. 2) Go through the standard process of users requesting module ownership, granting them ownership via the admin function, and then have them upload their types as they did in the prod db. Shortcuts are pretty much what I outlined before: 1) Grab all the most recent modules from the workspace service, a test team member requests ownership of all modules, approves via the admin function, and loads all the modules. This will not load older versions of modules, which may or may not be ok. 2) Give all the users admin access so they can approve their own modules. See https://atlassian.kbase.us/browse/WOR-80 for a potential long term solution. Long term, if you guys could outline how you'd like testing of types to work we can try to add the necessary functionality to the service. -g On 2/3/2014 9:17 AM, Gavin Price wrote: > > No, that's what I was saying in my response to Tom. You'd have to get > the types manually, one by one, and have the owners go through the > approval process on the test server and redo the uploads if you want > to duplicate the prod db. There are short cuts possible, though. I > need to deal with some moving stuff now so I'll outline those when I > get in. > > -g > Sent from my Cray XT30 > > On Feb 3, 2014 9:10 AM, "Shane Canon" <scanon(a)lbl.gov > <mailto:[email protected]>> wrote: > > > I think a dump from prod of the types and load to test makes > sense. This allows us to at least be somewhat in sync with the > production environment. I don't think it is feasible > to have all the data, but it is reasonable to have the types and > maybe some sample data. > > Gavin: Do APIs and commands presently exist to do this? > > --Shane > > > On Feb 2, 2014, at 7:56 PM, Gavin Price <gaprice(a)lbl.gov > <mailto:[email protected]>> wrote: > >> >> Sent from my Cray XT30 >> On Feb 2, 2014 5:07 PM, "Brettin, Thomas S." >> <brettin(a)cels.anl.gov <mailto:[email protected]>> wrote: >> > >> > -------- >> > WARNING: At least one of the links in the message below goes to >> an IP address (e.g.10.1.1.1), which could be malicious. To learn >> how to protect yourself, please go here: >> https://commons.lbl.gov/x/_591B >> > -------- >> > >> > We need a two step process. Dump all types in prod. Load onto test. >> That's not going to happen in the next couple weeks. >> >> > Seems backwards. >> It'll do for now. How we manage types in kbase needs an overall >> rethink, as we discussed. >> >> -g >> >> > >> > Sent from my Verizon Wireless 4G LTE DROID >> > >> > >> > Gavin Price <gaprice(a)lbl.gov <mailto:[email protected]>> wrote: >> > >> > You don't need to review them, and the mechanics of approving a >> type (or >> > a set of types) takes a couple seconds. I would just say that >> if a type >> > is available in the prod server it should be available in the test >> > server, and the prod server can act as the list of approved >> types. All >> > the test team needs to do is see if the requester owns the type >> in the >> > prod server and if so, approve it on the test server. This could >> > probably be scripted easily. >> > >> > > Why is type registration a risky operation? >> > Giving all users admin privileges on the test workspace service >> is what >> > I was saying seems dangerous. >> > >> > -g >> > >> > On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: >> > > -------- >> > > WARNING: At least one of the links in the message below goes >> to an IP address (e.g. 10.1.1.1), which could be malicious. To >> learn how to protect yourself, please go here: >> https://commons.lbl.gov/x/_591B >> > > -------- >> > > >> > > I don't think the test team has the bandwidth to review and >> approve types. I wouldn't even know where to start with criteria. >> > > >> > > Why is type registration a risky operation? >> > > >> > > --Dan >> > > >> > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov >> <mailto:[email protected]>> wrote: >> > >> >> > >> Firstly, you may want to check out >> > >> >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… >> > >> >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… >> > >> >> > >> These will allow you to translate type names between >> services even if the types were not loaded in the same order. >> Generally I'd recommend services use the MD5 style type in their >> code/settings and translate them to human readable types on startup. >> > >> >> > >> There are a few options that are possible: >> > >> >> > >> 1) The test team works with each module author to get the >> appropriate types installed in the test workspace. That means the >> module author needs to request the module name on the test server >> and a test team rep needs to approve that request via the admin >> interface. Then the module author is free to upload types as >> necessary. >> > >> >> > >> 2) The test team sets up a ws admin account and provides the >> credentials to testers and module authors, who can then approve >> their own modules. This seems a bit dangerous and I probably >> wouldn't recommend it >> > >> >> > >> 3) The test team owns all the modules in the test ws and >> installs any necessary types for testing themselves. >> > >> >> > >> -g >> > >> >> > >> >> > >>> On 2/2/2014 8:57 AM, Shane Canon wrote: >> > >>> Alexey, >> > >>> >> > >>> You raise some interesting issues. I hadn't appreciated >> some of these complexities. I'm glad you included Gavin in the >> chain. I'm hoping he has >> > >>> some ideas. For testing purposes perhaps we could upload a >> snapshot of the production types into the test workspace. >> > >>> >> > >>> --Shane >> > >>> >> > >>> >> > >>>> On Feb 1, 2014, at 9:20 PM, Alexey >> Kazakov<aekazakov(a)lbl.gov <mailto:[email protected]>> wrote: >> > >>>> >> > >>>> Hi all, >> > >>>> >> > >>>> workspace service 0.1.x that we use now has several big >> differences from the previous workspace version. In particular, >> it requires registration of object types for every module to >> enable the objects to be saved in workspace, and only module's >> owner can register typespec for the module. Thus, several actions >> must be performed before workspace will be populated with data: >> > >>>> 1. Module's owner requests an authorship of the module >> (Workspace.request_module_ownership function). >> > >>>> 2. Workspace administrator approves the request (workspace >> service WILL NOT notify module's owner about it). This step >> prevents implementation of the whole process as a single script. >> > >>>> 3. Module's owner registers typespec file and defines a >> list of types that will be available in workspace >> (Workspace.register_typespec function). At this step, typespec >> from another workspace can be copied by >> Workspace.register_typespec_copy function. >> > >>>> 4. Module's owner releases module for general use of its >> type (Workspace.release_module). >> > >>>> >> > >>>> These four steps can be performed once, as a preparation >> of a new workspace installation for data loading. But version >> numbers for all registered data types will be 1.0, and I >> anticipate a potential source of problems here for modules that >> use previous versions of data types. I do not use hard-coded >> version numbers in meme, cmonkey and inferelator services, but >> some other modules may be affected. >> > >>>> >> > >>>> If execution of a module depends on other modules or their >> data types, their typespec files must be registered too. For >> instance, meme module depends on types from KBaseSequences, >> cmonkey depends on types from MEME, KBaseExpression, >> KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on >> types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. >> > >>>> >> > >>>> Ideally, deployment of workspace 0.1.x in test environment >> should include requests of authorship for all modules currently >> registered in production workspace, with subsequent import of all >> typespecs (including all versions of typespec) in a proper order >> from production workspace. This will re-create all versions of >> data types from production workspace to ensure identical behavior >> of modules in test and production environments. What you think >> about it? >> > >>>> >> > >>>> Best, >> > >>>> Alexey >> > >>>> >> > >>>> >> > >>>> >> > >>>> >> > >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: >> > >>>>> I'm adding the release-team to the thread so they are in >> the loop. >> > >>>>> >> > >>>>> The test workspace is a standard deployment in the test >> infrastructure. The only difference is it isn't populated with >> the data that is in the production >> > >>>>> instance. In general, it would be best if the tests >> could assume that there isn't any data pre-loaded into a data >> store. So if it requires some data in >> > >>>>> the workspace, it should populate it as part of the test. >> > >>>>> >> > >>>>> In looking through the modules I see that there are >> various references to host in 140.84.*.* and 140.85.*.*. Are >> these "production" services? >> > >>>>> I'm worried that what we are deploying is actually >> contacting test services on Magellan. Also, we would prefer >> that URLs not be hard coded >> > >>>>> in the source code, but instead be read from the >> deployment configuration file. Right now it appears that all of >> this information is hard-coded >> > >>>>> in the java files, and we would need to recompile and >> redeploy if we needed to change a URL. >> > >>>>> >> > >>>>> By the way, my workaround for the type compiler issue is >> that I have add the dev branch of the type compiler as a >> submodule. This isn't ideal but >> > >>>>> at least provides a way for us to move forward. I will >> try to create a branch with these changes so that you can review >> them and merge them in >> > >>>>> if they seem reasonable. Another option would be for you >> to add the generated code to the module repo and only run the >> type comp when it >> > >>>>> is necessary (i.e. a major change in the type compiler or >> an API change). >> > >>>>> >> > >>>>> --Shane >> > >>>>> >> > >>>>> >> > >>>>> >> > >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey >> Kazakov<aekazakov(a)lbl.gov <mailto:[email protected]>> wrote: >> > >>>>>> >> > >>>>>> Hi Shane, >> > >>>>>> >> > >>>>>> cmonkey and inferelator tests should pass because these >> services call AWE client which is doing all calculations. Tests >> of three functions of meme that call AWE client passed too. I am >> trying to figure out how to deal with other meme functions that >> run MEME locally. How the workspace service was configured for >> testing environment? Were modules and object types from >> production WS made available for the testing WS, or this is only >> a new clean installation? >> > >>>>>> >> > >>>>>> Thank you, >> > >>>>>> Alexey >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: >> > >>>>>>> Pavel, >> > >>>>>>> >> > >>>>>>> Thanks for pointing me to Alexey. I apologize that I >> keep forgetting who was working on this. >> > >>>>>>> >> > >>>>>>> BTW, I have inferelator up and running. It seemed to >> pass the tests easily. There doesn't >> > >>>>>>> appear to be any code in bambi (just a spec file). So >> there isn't anything to deploy there yet. >> > >>>>>>> >> > >>>>>>> So the only one that we need to figure out is meme. >> This appears to just be due to testing >> > >>>>>>> against a workspace service that isn't the production >> one. So it is probably missing some >> > >>>>>>> objects that are used in the test. I would like to >> have a test I can run in the test environment >> > >>>>>>> just for completeness. But I think we are close. >> > >>>>>>> >> > >>>>>>> --Shane >> > >>>>>>> >> > >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel >> Novichkov<psnovichkov(a)lbl.gov <mailto:[email protected]>> wrote: >> > >>>>>>>> >> > >>>>>>>> Hi Shane, >> > >>>>>>>> >> > >>>>>>>> Alexey (cc-ed) was the primary developer of meme, >> cmnokey and inferelator services. Please talk to Alexey. >> > >>>>>>>> >> > >>>>>>>> Thanks, >> > >>>>>>>> P. >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane >> Canon<scanon(a)lbl.gov <mailto:[email protected]>> wrote: >> > >>>>>>>>> >> > >>>>>>>>> Pavel, >> > >>>>>>>>> >> > >>>>>>>>> I'm working through the services. I have cmonkey and >> meme deployed. I've noticed a few things that we will need to >> fix, but I'm working around for >> > >>>>>>>>> now. Some of these are artifacts due to we no longer >> install into /kb/deployment. So if the code or tests assume >> that, then it can cause problems. >> > >>>>>>>>> We also install as a non-root user. So this can >> sometime lead to problems too. If you or Roman grep through the >> code and look for references >> > >>>>>>>>> to /kb/deployment, then you will see what needs to be >> addressed. I also noticed that the tests have hard coded >> references for a magellan host. >> > >>>>>>>>> So that needs to be fixed. >> > >>>>>>>>> >> > >>>>>>>>> After deploying meme, I am seeing the following >> failures when I run make test. I could use some help to >> understand how to deal with these. >> > >>>>>>>>> >> > >>>>>>>>> For now, I will work on bambi and inferelator. >> > >>>>>>>>> >> > >>>>>>>>> --Shane >> > >>>>>>>>> >> > >>>>>>>>> >> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ >> make test >> > >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: >> overriding commands for target `deploy-libs' >> > >>>>>>>>> Makefile:47: warning: ignoring old commands for >> target `deploy-libs' >> > >>>>>>>>> # run each test >> > >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl >> test/script_tests-command-line.t ; \ >> > >>>>>>>>> if [ $? -ne 0 ] ; then \ >> > >>>>>>>>> exit 1 ; \ >> > >>>>>>>>> fi \ >> > >>>>>>>>> >> > >>>>>>>>> 1..24 >> > >>>>>>>>> ok 1 - creating Test::Cmd object for >> find_motifs_with_meme_from_ws >> > >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws >> > >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >> > >>>>>>>>> # at test/script_tests-command-line.t line 32. >> > >>>>>>>>> MemeRunResult ID: >> {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id >> cannot be null or the empty >> string","error":"us.kbase.common.service.ServerException: id >> cannot be null or the empty string\n\tat >> us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat >> us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat >> us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat >> us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat >> us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat >> us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat >> us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat >> >> > su >> > >> n.reflect.Deleg >> > >>>>>>>>> >> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat >> java.lang.reflect.Method.invoke(Method.java:606)\n\tat >> us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat >> javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat >> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat >> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat >> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat >> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat >> org.apache.catalina.connector.CoyoteA >> > da >> > >> pter.service(Coy >> > >>>>>>>>> oteAdapter.java:231)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat >> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat >> com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat >> com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat >> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat >> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat >> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat >> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat >> com.sun.griz >> > zl >> > >> y.SelectionKeyCo >> > >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat >> com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat >> java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >> > >>>>>>>>> >> > >>>>>>>>> not ok 3 - MEME run successfuly >> > >>>>>>>>> # Failed test 'MEME run successfuly' >> > >>>>>>>>> # at test/script_tests-command-line.t line 35. >> > >>>>>>>>> ok 4 - creating Test::Cmd object for >> find_motifs_with_meme_job_from_ws >> > >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >> > >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> ok 6 - MEME run successfuly >> > >>>>>>>>> ok 7 - creating Test::Cmd object for >> get_pspm_collection_from_meme_result_from_ws >> > >>>>>>>>> not ok 8 - Running >> get_pspm_collection_from_meme_result_from_ws >> > >>>>>>>>> # Failed test 'Running >> get_pspm_collection_from_meme_result_from_ws' >> > >>>>>>>>> # at test/script_tests-command-line.t line 50. >> > >>>>>>>>> MemePSPMCollection ID: >> {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id >> cannot be null or the empty >> string","error":"us.kbase.common.service.ServerException: id >> cannot be null or the empty string\n\tat >> us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat >> us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat >> us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat >> us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat >> us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat >> us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat >> us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat >> sun.reflect.NativeMethodAccesso >> > rI >> > >> mpl.invoke(Nati >> > >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat >> java.lang.reflect.Method.invoke(Method.java:606)\n\tat >> us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat >> javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat >> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat >> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat >> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat >> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. >> > ja >> > >> va:331)\n\tat or >> > >>>>>>>>> >> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat >> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat >> com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat >> com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat >> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat >> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat >> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat >> com.sun.grizzly.ProtocolChainContextTask.doCall >> > (P >> > >> rotocolChainCont >> > >>>>>>>>> extTask.java:54)\n\tat >> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat >> com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat >> java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >> > >>>>>>>>> >> > >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly >> > >>>>>>>>> # Failed test 'MEME PSPM collection generated >> successfuly' >> > >>>>>>>>> # at test/script_tests-command-line.t line 53. >> > >>>>>>>>> ok 10 - creating Test::Cmd object for >> get_pspm_collection_from_meme_result_job_from_ws >> > >>>>>>>>> not ok 11 - Running >> get_pspm_collection_from_meme_result_job_from_ws >> > >>>>>>>>> # Failed test 'Running >> get_pspm_collection_from_meme_result_job_from_ws' >> > >>>>>>>>> # at test/script_tests-command-line.t line 59. >> > >>>>>>>>> Job ID: >> {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult >> download error","error":"java.lang.Exception: MemeRunResult >> download error\n\tat >> us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat >> us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat >> us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat >> java.lang.reflect.Method.invoke(Method.java:606)\n\tat >> us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat >> javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:77 >> > 0) >> > >> \n\tat org.apac >> > >>>>>>>>> >> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat >> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat >> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat >> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat >> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat >> com.sun.grizzly.http.ProcessorTask >> > .d >> > >> oProcess(Process >> > >>>>>>>>> orTask.java:757)\n\tat >> com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat >> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat >> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat >> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat >> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat >> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat >> com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat >> java.lang.Thread.run(Thread.java:744)\n"},"id":"6 >> > 56 >> > >> 565433923738"} >> > >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly >> > >>>>>>>>> ok 13 - creating Test::Cmd object for >> compare_motifs_with_tomtom_by_collection_from_ws >> > >>>>>>>>> not ok 14 - Running >> compare_motifs_with_tomtom_by_collection_from_ws >> > >>>>>>>>> # Failed test 'Running >> compare_motifs_with_tomtom_by_collection_from_ws' >> > >>>>>>>>> # at test/script_tests-command-line.t line 69. >> > >>>>>>>>> TomtomRunResult ID: Failed to acquire token: >> Failed to get auth token: Failed to query Globus Online: 500 read >> timeout at >> /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line >> 574. >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> not ok 15 - TOMTOM run successfuly >> > >>>>>>>>> # Failed test 'TOMTOM run successfuly' >> > >>>>>>>>> # at test/script_tests-command-line.t line 72. >> > >>>>>>>>> ok 16 - creating Test::Cmd object for >> compare_motifs_with_tomtom_job_by_collection_from_ws >> > >>>>>>>>> ok 17 - Running >> compare_motifs_with_tomtom_job_by_collection_from_ws >> > >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> ok 18 - TOMTOM run successfuly >> > >>>>>>>>> ok 19 - creating Test::Cmd object for >> find_sites_with_mast_by_collection_from_ws >> > >>>>>>>>> not ok 20 - Running >> find_sites_with_mast_by_collection_from_ws >> > >>>>>>>>> # Failed test 'Running >> find_sites_with_mast_by_collection_from_ws' >> > >>>>>>>>> # at test/script_tests-command-line.t line 87. >> > >>>>>>>>> MastRunResult ID: >> {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id >> cannot be null or the empty >> string","error":"us.kbase.common.service.ServerException: id >> cannot be null or the empty string\n\tat >> us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat >> us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat >> us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat >> us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat >> us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat >> us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat >> us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat >> sun.reflect.NativeMethodAccessorImpl. >> > in >> > >> voke(NativeMeth >> > >>>>>>>>> odAccessorImpl.java:57)\n\tat >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat >> java.lang.reflect.Method.invoke(Method.java:606)\n\tat >> us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat >> javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat >> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat >> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat >> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat >> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 >> > 31 >> > >> )\n\tat org.apac >> > >>>>>>>>> >> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat >> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat >> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat >> com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat >> com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat >> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat >> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat >> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat >> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat >> com.sun.grizzly.ProtocolChainContextTask.doCall(Proto >> > co >> > >> lChainContextTas >> > >>>>>>>>> k.java:54)\n\tat >> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat >> com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat >> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat >> java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >> > >>>>>>>>> >> > >>>>>>>>> not ok 21 - MAST run successfuly >> > >>>>>>>>> # Failed test 'MAST run successfuly' >> > >>>>>>>>> # at test/script_tests-command-line.t line 90. >> > >>>>>>>>> ok 22 - creating Test::Cmd object for >> find_sites_with_mast_job_by_collection_from_ws >> > >>>>>>>>> ok 23 - Running >> find_sites_with_mast_job_by_collection_from_ws >> > >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 >> > >>>>>>>>> >> > >>>>>>>>> >> > >>>>>>>>> ok 24 - MAST run successfuly >> > >>>>>>>>> # Looks like you failed 9 tests of 24. >> > >>>>>>>>> make: *** [test-scripts] Error 1 >> > >> _______________________________________________ >> > >> Release-team mailing list >> > >> Release-team(a)lists.kbase.us <mailto:[email protected]> >> > >> https://lists.kbase.us/mailman/listinfo/release-team >> > > >> > >> > _______________________________________________ >> > Release-team mailing list >> > Release-team(a)lists.kbase.us <mailto:[email protected]> >> > https://lists.kbase.us/mailman/listinfo/release-team >> >> _______________________________________________ >> Release-team mailing list >> Release-team(a)lists.kbase.us <mailto:[email protected]> >> https://lists.kbase.us/mailman/listinfo/release-team >

1 0

Re: [Release-team] KBase Module status as of Feb. 3
by Robert Olson 03 Feb '14

03 Feb '14

OK, thanks. —bob On Feb 3, 2014, at 11:22 AM, Brettin, Thomas S. <brettin(a)cels.anl.gov> wrote: > Bob, Your part in this is done. > > I was saying that kb_seed is coupled to the new version of the central store, and the cutover to production for the two is coordinated. > > Tom > > > On Feb 3, 2014, at 8:57 AM, Robert Olson <olson(a)mcs.anl.gov> wrote: > >> Commitment from whom? I don’t know anything about this. >> >> On Feb 3, 2014, at 8:52 AM, Land, Miriam L. <landml(a)ornl.gov> wrote: >> >>> kb_seed/Central Store – we have commitment that it will be updated today based on new database structure >> >> _______________________________________________ >> Release-team mailing list >> Release-team(a)lists.kbase.us >> https://lists.kbase.us/mailman/listinfo/release-team >

1 0

Re: [Release-team] KBase Module status as of Feb. 3
by Brettin, Thomas S. 03 Feb '14

03 Feb '14

Miriam, For those modules w/o contingency plans, we're not oblibated to do anything. Tom On Feb 3, 2014, at 8:52 AM, "Land, Miriam L." <landml(a)ornl.gov> wrote: > Release Team, > > From the spreadsheet, here is what I see as the status: > > Assembly – had git commits on Jan. 27 and 28 – I assume it is ready for deploy/test > KBase FBA modeling – Had a commit yesterday – I assume it is ready for deploy/test > gene_conservation – had commits on Jan. 31 – I assume it is ready for deploy/test > genome_annotation – had commits on Jan. 31 – I assume it is ready for deploy/test > Meme, inferelator, cmonkey – deployed and tested over the weekend. Ready for documentation testing > Search – has been tested over the weekend and problems appear to be with configuration, not the code > kb_seed/Central Store – we have commitment that it will be updated today based on new database structure > > Expression, Network, MAK, and bambi have asked for an extension to Tuesday sometime on the west coast. > > Are we prepared to act on the consequences of delayed modules without contingency plans? > > Tom, are there any waivers? > > Miriam > > From: <Land>, Miriam Land <landml(a)ornl.gov<mailto:[email protected]>> > Date: Friday, January 31, 2014 10:23 AM > To: KBase-devel <kbase-devel(a)lists.kbase.us<mailto:[email protected]>> > Subject: [Kbase-devel] KBase Module status as of Jan. 31, 2014 - Deadline is near > > All, > > DROP DEAD DAY APPROACHES – All modules must be checked into git by 8am Eastern time Monday Feb. 3. > > Updates since yesterday – > > * id_map was unit tested had passed documentation review > * AWE, Shock, and workspace_deluxe have had independent testing and no longer need black box testing. > * Search is in deploy and test on the test VM > > As always, I welcome corrections and updates to the information below. > > ---------------------------------------------------------------------------------------------------------------------------- > Modules expecting to update after Jan. 27 > > * Assembly - implementing an option for integration with the narratives. Expect completion by Feb. 3. > * id_map > * KBase Expression – needs DB connection and config updates > * gene_conservation has end-of-month target > * IRIS – What is in testing is already in production. There may be a new version before Feb. 3. > * KB_seed update will be necessary if production access to the new database schema is required > * Networks dependent on data load and has end-of-month target > * Modeling will have a HUGE update by Thurs. (Jan. 30) > > Modules with a submitted contingency plan > > * Search > > Modules ready for production deployment > > * Authentication and Authorization Client > * CBD and Probabilistic Annotation - will need black box testing at some point > * Communities API – passed tests except for two that need to be withdrawn (Andreas and Travis) > * Genotyping Service > * Handle Service (on t6) - internal service - will need black box testing at some point > * Interlog projection - will need black box testing at some point > * m5nr - (Andreas and Travis) - also will need black box testing at some point > * Workspace_deluxe – > * Awe Service – internal service > * Shock Service – internal service > > Modules ready to deploy to testing VM > > * genome_annotation – (Bob O. 1/30) > > Modules currently in testing > > * Search – (Matt Henderson 1/27) > > Modules ready for review of scripts, documentation, and release notes > > * none > > Modules needing updates to release notes > > * KBase Expression Services > * Network service > * AWE and Shock – release notes desirable but not mandatory > * All modules that haven't been tested yet > > Modules waiting on database updates > > * FBA modeling - transitioning to the new workspace (Chris H.) > * KBase Expression – need DB connection and config updates afterwards (confirmed 1/27) (Jason) > > Modules waiting on a developer > > * Assembly - option for the narrative will be added > * Bambi, Cmonkey, inferelator, meme and possibly MAK will be ready 1/28 > * Gene conservation – development continuing (end of month confirmed 1/22) (Gang Fang) - will need black box testing at some point > * kb_seed – update to new database schema (Bob O. 1/27) > * Networks - development continuing and data load dependency, by Jan. 31 (confirmed 1/22) (Pavel) > > Modules Withdrawn from the formal build process > > * Functional Ortholog Predictor – delay until after Feb. demo (1/23/14 Fei He) > * Landing Pages (Parmvir) - Will not be part of this build cycle (S. Chan 1/23/14) > * Narrative (Steve Chan) - Will not be part of this build cycle (S. Chan 1/23/14) > > Miriam > _______________________________________________ > Kbase-devel mailing list > Kbase-devel(a)lists.kbase.us<mailto:[email protected]> > https://lists.kbase.us/mailman/listinfo/kbase-devel > > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] KBase Module status as of Feb. 3
by Brettin, Thomas S. 03 Feb '14

03 Feb '14

Bob, Your part in this is done. I was saying that kb_seed is coupled to the new version of the central store, and the cutover to production for the two is coordinated. Tom On Feb 3, 2014, at 8:57 AM, Robert Olson <olson(a)mcs.anl.gov<mailto:[email protected]>> wrote: Commitment from whom? I don’t know anything about this. On Feb 3, 2014, at 8:52 AM, Land, Miriam L. <landml(a)ornl.gov<mailto:[email protected]>> wrote: kb_seed/Central Store – we have commitment that it will be updated today based on new database structure _______________________________________________ Release-team mailing list Release-team(a)lists.kbase.us<mailto:[email protected]> https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Gavin Price 03 Feb '14

03 Feb '14

No, that's what I was saying in my response to Tom. You'd have to get the types manually, one by one, and have the owners go through the approval process on the test server and redo the uploads if you want to duplicate the prod db. There are short cuts possible, though. I need to deal with some moving stuff now so I'll outline those when I get in. -g Sent from my Cray XT30 On Feb 3, 2014 9:10 AM, "Shane Canon" <scanon(a)lbl.gov> wrote: > > I think a dump from prod of the types and load to test makes sense. This > allows us to at least be somewhat in sync with the production environment. > I don't think it is feasible > to have all the data, but it is reasonable to have the types and maybe > some sample data. > > Gavin: Do APIs and commands presently exist to do this? > > --Shane > > > On Feb 2, 2014, at 7:56 PM, Gavin Price <gaprice(a)lbl.gov> wrote: > > > Sent from my Cray XT30 > On Feb 2, 2014 5:07 PM, "Brettin, Thomas S." <brettin(a)cels.anl.gov> wrote: > > > > -------- > > WARNING: At least one of the links in the message below goes to an IP > address (e.g.10.1.1.1), which could be malicious. To learn how to protect > yourself, please go here: https://commons.lbl.gov/x/_591B > > -------- > > > > We need a two step process. Dump all types in prod. Load onto test. > That's not going to happen in the next couple weeks. > > > Seems backwards. > It'll do for now. How we manage types in kbase needs an overall rethink, > as we discussed. > > -g > > > > > Sent from my Verizon Wireless 4G LTE DROID > > > > > > Gavin Price <gaprice(a)lbl.gov> wrote: > > > > You don't need to review them, and the mechanics of approving a type (or > > a set of types) takes a couple seconds. I would just say that if a type > > is available in the prod server it should be available in the test > > server, and the prod server can act as the list of approved types. All > > the test team needs to do is see if the requester owns the type in the > > prod server and if so, approve it on the test server. This could > > probably be scripted easily. > > > > > Why is type registration a risky operation? > > Giving all users admin privileges on the test workspace service is what > > I was saying seems dangerous. > > > > -g > > > > On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > > > -------- > > > WARNING: At least one of the links in the message below goes to an IP > address (e.g. 10.1.1.1), which could be malicious. To learn how to protect > yourself, please go here: https://commons.lbl.gov/x/_591B > > > -------- > > > > > > I don't think the test team has the bandwidth to review and approve > types. I wouldn't even know where to start with criteria. > > > > > > Why is type registration a risky operation? > > > > > > --Dan > > > > > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov> wrote: > > >> > > >> Firstly, you may want to check out > > >> > http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > > >> > http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > > >> > > >> These will allow you to translate type names between services even if > the types were not loaded in the same order. Generally I'd recommend > services use the MD5 style type in their code/settings and translate them > to human readable types on startup. > > >> > > >> There are a few options that are possible: > > >> > > >> 1) The test team works with each module author to get the appropriate > types installed in the test workspace. That means the module author needs > to request the module name on the test server and a test team rep needs to > approve that request via the admin interface. Then the module author is > free to upload types as necessary. > > >> > > >> 2) The test team sets up a ws admin account and provides the > credentials to testers and module authors, who can then approve their own > modules. This seems a bit dangerous and I probably wouldn't recommend it > > >> > > >> 3) The test team owns all the modules in the test ws and installs any > necessary types for testing themselves. > > >> > > >> -g > > >> > > >> > > >>> On 2/2/2014 8:57 AM, Shane Canon wrote: > > >>> Alexey, > > >>> > > >>> You raise some interesting issues. I hadn't appreciated some of > these complexities. I'm glad you included Gavin in the chain. I'm hoping > he has > > >>> some ideas. For testing purposes perhaps we could upload a snapshot > of the production types into the test workspace. > > >>> > > >>> --Shane > > >>> > > >>> > > >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> > wrote: > > >>>> > > >>>> Hi all, > > >>>> > > >>>> workspace service 0.1.x that we use now has several big differences > from the previous workspace version. In particular, it requires > registration of object types for every module to enable the objects to be > saved in workspace, and only module's owner can register typespec for the > module. Thus, several actions must be performed before workspace will be > populated with data: > > >>>> 1. Module's owner requests an authorship of the module > (Workspace.request_module_ownership function). > > >>>> 2. Workspace administrator approves the request (workspace service > WILL NOT notify module's owner about it). This step prevents implementation > of the whole process as a single script. > > >>>> 3. Module's owner registers typespec file and defines a list of > types that will be available in workspace (Workspace.register_typespec > function). At this step, typespec from another workspace can be copied by > Workspace.register_typespec_copy function. > > >>>> 4. Module's owner releases module for general use of its type > (Workspace.release_module). > > >>>> > > >>>> These four steps can be performed once, as a preparation of a new > workspace installation for data loading. But version numbers for all > registered data types will be 1.0, and I anticipate a potential source of > problems here for modules that use previous versions of data types. I do > not use hard-coded version numbers in meme, cmonkey and inferelator > services, but some other modules may be affected. > > >>>> > > >>>> If execution of a module depends on other modules or their data > types, their typespec files must be registered too. For instance, meme > module depends on types from KBaseSequences, cmonkey depends on types from > MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and > inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression > typespecs. > > >>>> > > >>>> Ideally, deployment of workspace 0.1.x in test environment should > include requests of authorship for all modules currently registered in > production workspace, with subsequent import of all typespecs (including > all versions of typespec) in a proper order from production workspace. This > will re-create all versions of data types from production workspace to > ensure identical behavior of modules in test and production environments. > What you think about it? > > >>>> > > >>>> Best, > > >>>> Alexey > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: > > >>>>> I'm adding the release-team to the thread so they are in the loop. > > >>>>> > > >>>>> The test workspace is a standard deployment in the test > infrastructure. The only difference is it isn't populated with the data > that is in the production > > >>>>> instance. In general, it would be best if the tests could assume > that there isn't any data pre-loaded into a data store. So if it requires > some data in > > >>>>> the workspace, it should populate it as part of the test. > > >>>>> > > >>>>> In looking through the modules I see that there are various > references to host in 140.84.*.* and 140.85.*.*. Are these "production" > services? > > >>>>> I'm worried that what we are deploying is actually contacting test > services on Magellan. Also, we would prefer that URLs not be hard coded > > >>>>> in the source code, but instead be read from the deployment > configuration file. Right now it appears that all of this information is > hard-coded > > >>>>> in the java files, and we would need to recompile and redeploy if > we needed to change a URL. > > >>>>> > > >>>>> By the way, my workaround for the type compiler issue is that I > have add the dev branch of the type compiler as a submodule. This isn't > ideal but > > >>>>> at least provides a way for us to move forward. I will try to > create a branch with these changes so that you can review them and merge > them in > > >>>>> if they seem reasonable. Another option would be for you to add > the generated code to the module repo and only run the type comp when it > > >>>>> is necessary (i.e. a major change in the type compiler or an API > change). > > >>>>> > > >>>>> --Shane > > >>>>> > > >>>>> > > >>>>> > > >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> > wrote: > > >>>>>> > > >>>>>> Hi Shane, > > >>>>>> > > >>>>>> cmonkey and inferelator tests should pass because these services > call AWE client which is doing all calculations. Tests of three functions > of meme that call AWE client passed too. I am trying to figure out how to > deal with other meme functions that run MEME locally. How the workspace > service was configured for testing environment? Were modules and object > types from production WS made available for the testing WS, or this is only > a new clean installation? > > >>>>>> > > >>>>>> Thank you, > > >>>>>> Alexey > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: > > >>>>>>> Pavel, > > >>>>>>> > > >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep > forgetting who was working on this. > > >>>>>>> > > >>>>>>> BTW, I have inferelator up and running. It seemed to pass the > tests easily. There doesn't > > >>>>>>> appear to be any code in bambi (just a spec file). So there > isn't anything to deploy there yet. > > >>>>>>> > > >>>>>>> So the only one that we need to figure out is meme. This > appears to just be due to testing > > >>>>>>> against a workspace service that isn't the production one. So > it is probably missing some > > >>>>>>> objects that are used in the test. I would like to have a test > I can run in the test environment > > >>>>>>> just for completeness. But I think we are close. > > >>>>>>> > > >>>>>>> --Shane > > >>>>>>> > > >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov< > psnovichkov(a)lbl.gov> wrote: > > >>>>>>>> > > >>>>>>>> Hi Shane, > > >>>>>>>> > > >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and > inferelator services. Please talk to Alexey. > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> P. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> > wrote: > > >>>>>>>>> > > >>>>>>>>> Pavel, > > >>>>>>>>> > > >>>>>>>>> I'm working through the services. I have cmonkey and meme > deployed. I've noticed a few things that we will need to fix, but I'm > working around for > > >>>>>>>>> now. Some of these are artifacts due to we no longer install > into /kb/deployment. So if the code or tests assume that, then it can > cause problems. > > >>>>>>>>> We also install as a non-root user. So this can sometime lead > to problems too. If you or Roman grep through the code and look for > references > > >>>>>>>>> to /kb/deployment, then you will see what needs to be > addressed. I also noticed that the tests have hard coded references for a > magellan host. > > >>>>>>>>> So that needs to be fixed. > > >>>>>>>>> > > >>>>>>>>> After deploying meme, I am seeing the following failures when > I run make test. I could use some help to understand how to deal with > these. > > >>>>>>>>> > > >>>>>>>>> For now, I will work on bambi and inferelator. > > >>>>>>>>> > > >>>>>>>>> --Shane > > >>>>>>>>> > > >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ > make test > > >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding > commands for target `deploy-libs' > > >>>>>>>>> Makefile:47: warning: ignoring old commands for target > `deploy-libs' > > >>>>>>>>> # run each test > > >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl > test/script_tests-command-line.t ; \ > > >>>>>>>>> if [ $? -ne 0 ] ; then \ > > >>>>>>>>> exit 1 ; \ > > >>>>>>>>> fi \ > > >>>>>>>>> > > >>>>>>>>> 1..24 > > >>>>>>>>> ok 1 - creating Test::Cmd object for > find_motifs_with_meme_from_ws > > >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws > > >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 32. > > >>>>>>>>> MemeRunResult ID: > {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id > cannot be null or the empty > string","error":"us.kbase.common.service.ServerException: id cannot be null > or the empty string\n\tat > us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat > us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat > us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat > us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat > us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat > us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat > us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat > > > su > > >> n.reflect.Deleg > > >>>>>>>>> > atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat > java.lang.reflect.Method.invoke(Method.java:606)\n\tat > us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat > org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat > org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat > org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat > org.apache.catalina.connector.CoyoteA > > da > > >> pter.service(Coy > > >>>>>>>>> oteAdapter.java:231)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat > com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat > com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat > com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat > com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat > com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat > com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat > com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat > com.sun.griz > > zl > > >> y.SelectionKeyCo > > >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat > com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat > java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} > > >>>>>>>>> > > >>>>>>>>> not ok 3 - MEME run successfuly > > >>>>>>>>> # Failed test 'MEME run successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 35. > > >>>>>>>>> ok 4 - creating Test::Cmd object for > find_motifs_with_meme_job_from_ws > > >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws > > >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ok 6 - MEME run successfuly > > >>>>>>>>> ok 7 - creating Test::Cmd object for > get_pspm_collection_from_meme_result_from_ws > > >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws > > >>>>>>>>> # Failed test 'Running > get_pspm_collection_from_meme_result_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 50. > > >>>>>>>>> MemePSPMCollection ID: > {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id > cannot be null or the empty > string","error":"us.kbase.common.service.ServerException: id cannot be null > or the empty string\n\tat > us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat > us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat > us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat > us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat > us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat > us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat > us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat > sun.reflect.NativeMethodAccesso > > rI > > >> mpl.invoke(Nati > > >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat > java.lang.reflect.Method.invoke(Method.java:606)\n\tat > us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat > org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat > org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat > org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. > > ja > > >> va:331)\n\tat or > > >>>>>>>>> > g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat > com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat > com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat > com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat > com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat > com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat > com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat > com.sun.grizzly.ProtocolChainContextTask.doCall > > (P > > >> rotocolChainCont > > >>>>>>>>> extTask.java:54)\n\tat > com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat > com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat > java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} > > >>>>>>>>> > > >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly > > >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 53. > > >>>>>>>>> ok 10 - creating Test::Cmd object for > get_pspm_collection_from_meme_result_job_from_ws > > >>>>>>>>> not ok 11 - Running > get_pspm_collection_from_meme_result_job_from_ws > > >>>>>>>>> # Failed test 'Running > get_pspm_collection_from_meme_result_job_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 59. > > >>>>>>>>> Job ID: > {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult > download error","error":"java.lang.Exception: MemeRunResult download > error\n\tat > us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat > us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat > us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat > java.lang.reflect.Method.invoke(Method.java:606)\n\tat > us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:77 > > 0) > > >> \n\tat org.apac > > >>>>>>>>> > he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat > org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat > org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat > com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat > com.sun.grizzly.http.ProcessorTask > > .d > > >> oProcess(Process > > >>>>>>>>> orTask.java:757)\n\tat > com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat > com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat > com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat > com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat > com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat > com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat > com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat > java.lang.Thread.run(Thread.java:744)\n"},"id":"6 > > 56 > > >> 565433923738"} > > >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly > > >>>>>>>>> ok 13 - creating Test::Cmd object for > compare_motifs_with_tomtom_by_collection_from_ws > > >>>>>>>>> not ok 14 - Running > compare_motifs_with_tomtom_by_collection_from_ws > > >>>>>>>>> # Failed test 'Running > compare_motifs_with_tomtom_by_collection_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 69. > > >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get > auth token: Failed to query Globus Online: 500 read timeout at > /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> not ok 15 - TOMTOM run successfuly > > >>>>>>>>> # Failed test 'TOMTOM run successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 72. > > >>>>>>>>> ok 16 - creating Test::Cmd object for > compare_motifs_with_tomtom_job_by_collection_from_ws > > >>>>>>>>> ok 17 - Running > compare_motifs_with_tomtom_job_by_collection_from_ws > > >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ok 18 - TOMTOM run successfuly > > >>>>>>>>> ok 19 - creating Test::Cmd object for > find_sites_with_mast_by_collection_from_ws > > >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws > > >>>>>>>>> # Failed test 'Running > find_sites_with_mast_by_collection_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 87. > > >>>>>>>>> MastRunResult ID: > {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id > cannot be null or the empty > string","error":"us.kbase.common.service.ServerException: id cannot be null > or the empty string\n\tat > us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat > us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat > us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat > us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat > us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat > us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat > us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat > sun.reflect.NativeMethodAccessorImpl. > > in > > >> voke(NativeMeth > > >>>>>>>>> odAccessorImpl.java:57)\n\tat > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat > java.lang.reflect.Method.invoke(Method.java:606)\n\tat > us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat > javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat > org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat > org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat > org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat > org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 > > 31 > > >> )\n\tat org.apac > > >>>>>>>>> > he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat > com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat > com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat > com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat > com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat > com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat > com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat > com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat > com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat > com.sun.grizzly.ProtocolChainContextTask.doCall(Proto > > co > > >> lChainContextTas > > >>>>>>>>> k.java:54)\n\tat > com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat > com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat > com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat > java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} > > >>>>>>>>> > > >>>>>>>>> not ok 21 - MAST run successfuly > > >>>>>>>>> # Failed test 'MAST run successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 90. > > >>>>>>>>> ok 22 - creating Test::Cmd object for > find_sites_with_mast_job_by_collection_from_ws > > >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws > > >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ok 24 - MAST run successfuly > > >>>>>>>>> # Looks like you failed 9 tests of 24. > > >>>>>>>>> make: *** [test-scripts] Error 1 > > >> _______________________________________________ > > >> Release-team mailing list > > >> Release-team(a)lists.kbase.us > > >> https://lists.kbase.us/mailman/listinfo/release-team > > > > > > > _______________________________________________ > > Release-team mailing list > > Release-team(a)lists.kbase.us > > https://lists.kbase.us/mailman/listinfo/release-team > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team > > >

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Shane Canon 03 Feb '14

03 Feb '14

I think a dump from prod of the types and load to test makes sense. This allows us to at least be somewhat in sync with the production environment. I don't think it is feasible to have all the data, but it is reasonable to have the types and maybe some sample data. Gavin: Do APIs and commands presently exist to do this? --Shane On Feb 2, 2014, at 7:56 PM, Gavin Price <gaprice(a)lbl.gov> wrote: > > Sent from my Cray XT30 > On Feb 2, 2014 5:07 PM, "Brettin, Thomas S." <brettin(a)cels.anl.gov> wrote: > > > > -------- > > WARNING: At least one of the links in the message below goes to an IP address (e.g.10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > > -------- > > > > We need a two step process. Dump all types in prod. Load onto test. > That's not going to happen in the next couple weeks. > > > Seems backwards. > It'll do for now. How we manage types in kbase needs an overall rethink, as we discussed. > > -g > > > > > Sent from my Verizon Wireless 4G LTE DROID > > > > > > Gavin Price <gaprice(a)lbl.gov> wrote: > > > > You don't need to review them, and the mechanics of approving a type (or > > a set of types) takes a couple seconds. I would just say that if a type > > is available in the prod server it should be available in the test > > server, and the prod server can act as the list of approved types. All > > the test team needs to do is see if the requester owns the type in the > > prod server and if so, approve it on the test server. This could > > probably be scripted easily. > > > > > Why is type registration a risky operation? > > Giving all users admin privileges on the test workspace service is what > > I was saying seems dangerous. > > > > -g > > > > On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > > > -------- > > > WARNING: At least one of the links in the message below goes to an IP address (e.g. 10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > > > -------- > > > > > > I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. > > > > > > Why is type registration a risky operation? > > > > > > --Dan > > > > > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov> wrote: > > >> > > >> Firstly, you may want to check out > > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > > >> > > >> These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. > > >> > > >> There are a few options that are possible: > > >> > > >> 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. > > >> > > >> 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it > > >> > > >> 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. > > >> > > >> -g > > >> > > >> > > >>> On 2/2/2014 8:57 AM, Shane Canon wrote: > > >>> Alexey, > > >>> > > >>> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has > > >>> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. > > >>> > > >>> --Shane > > >>> > > >>> > > >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: > > >>>> > > >>>> Hi all, > > >>>> > > >>>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: > > >>>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). > > >>>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. > > >>>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. > > >>>> 4. Module's owner releases module for general use of its type (Workspace.release_module). > > >>>> > > >>>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. > > >>>> > > >>>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. > > >>>> > > >>>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? > > >>>> > > >>>> Best, > > >>>> Alexey > > >>>> > > >>>> > > >>>> > > >>>> > > >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: > > >>>>> I'm adding the release-team to the thread so they are in the loop. > > >>>>> > > >>>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production > > >>>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in > > >>>>> the workspace, it should populate it as part of the test. > > >>>>> > > >>>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? > > >>>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded > > >>>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded > > >>>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. > > >>>>> > > >>>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but > > >>>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in > > >>>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it > > >>>>> is necessary (i.e. a major change in the type compiler or an API change). > > >>>>> > > >>>>> --Shane > > >>>>> > > >>>>> > > >>>>> > > >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: > > >>>>>> > > >>>>>> Hi Shane, > > >>>>>> > > >>>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? > > >>>>>> > > >>>>>> Thank you, > > >>>>>> Alexey > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: > > >>>>>>> Pavel, > > >>>>>>> > > >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. > > >>>>>>> > > >>>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't > > >>>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. > > >>>>>>> > > >>>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing > > >>>>>>> against a workspace service that isn't the production one. So it is probably missing some > > >>>>>>> objects that are used in the test. I would like to have a test I can run in the test environment > > >>>>>>> just for completeness. But I think we are close. > > >>>>>>> > > >>>>>>> --Shane > > >>>>>>> > > >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: > > >>>>>>>> > > >>>>>>>> Hi Shane, > > >>>>>>>> > > >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. > > >>>>>>>> > > >>>>>>>> Thanks, > > >>>>>>>> P. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: > > >>>>>>>>> > > >>>>>>>>> Pavel, > > >>>>>>>>> > > >>>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for > > >>>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. > > >>>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references > > >>>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. > > >>>>>>>>> So that needs to be fixed. > > >>>>>>>>> > > >>>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. > > >>>>>>>>> > > >>>>>>>>> For now, I will work on bambi and inferelator. > > >>>>>>>>> > > >>>>>>>>> --Shane > > >>>>>>>>> > > >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test > > >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' > > >>>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' > > >>>>>>>>> # run each test > > >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ > > >>>>>>>>> if [ $? -ne 0 ] ; then \ > > >>>>>>>>> exit 1 ; \ > > >>>>>>>>> fi \ > > >>>>>>>>> > > >>>>>>>>> 1..24 > > >>>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws > > >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws > > >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 32. > > >>>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat > > su > > >> n.reflect.Deleg > > >>>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteA > > da > > >> pter.service(Coy > > >>>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.griz > > zl > > >> y.SelectionKeyCo > > >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} > > >>>>>>>>> > > >>>>>>>>> not ok 3 - MEME run successfuly > > >>>>>>>>> # Failed test 'MEME run successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 35. > > >>>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws > > >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws > > >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ok 6 - MEME run successfuly > > >>>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws > > >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws > > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 50. > > >>>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccesso > > rI > > >> mpl.invoke(Nati > > >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. > > ja > > >> va:331)\n\tat or > > >>>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall > > (P > > >> rotocolChainCont > > >>>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} > > >>>>>>>>> > > >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly > > >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 53. > > >>>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws > > >>>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws > > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 59. > > >>>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:77 > > 0) > > >> \n\tat org.apac > > >>>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask > > .d > > >> oProcess(Process > > >>>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"6 > > 56 > > >> 565433923738"} > > >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly > > >>>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws > > >>>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws > > >>>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 69. > > >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> not ok 15 - TOMTOM run successfuly > > >>>>>>>>> # Failed test 'TOMTOM run successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 72. > > >>>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws > > >>>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws > > >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ok 18 - TOMTOM run successfuly > > >>>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws > > >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws > > >>>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' > > >>>>>>>>> # at test/script_tests-command-line.t line 87. > > >>>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl. > > in > > >> voke(NativeMeth > > >>>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 > > 31 > > >> )\n\tat org.apac > > >>>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Proto > > co > > >> lChainContextTas > > >>>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} > > >>>>>>>>> > > >>>>>>>>> not ok 21 - MAST run successfuly > > >>>>>>>>> # Failed test 'MAST run successfuly' > > >>>>>>>>> # at test/script_tests-command-line.t line 90. > > >>>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws > > >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws > > >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> ok 24 - MAST run successfuly > > >>>>>>>>> # Looks like you failed 9 tests of 24. > > >>>>>>>>> make: *** [test-scripts] Error 1 > > >> _______________________________________________ > > >> Release-team mailing list > > >> Release-team(a)lists.kbase.us > > >> https://lists.kbase.us/mailman/listinfo/release-team > > > > > > > _______________________________________________ > > Release-team mailing list > > Release-team(a)lists.kbase.us > > https://lists.kbase.us/mailman/listinfo/release-team > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] KBase Module status as of Feb. 3
by Land, Miriam L. 03 Feb '14

03 Feb '14

Perhaps that was overstated. It was a note that Dan took on Thurs. Miriam From: Robert Olson <olson(a)mcs.anl.gov<mailto:[email protected]>> Date: Monday, February 3, 2014 9:57 AM To: Miriam Land <landml(a)ornl.gov<mailto:[email protected]>> Cc: "release-team(a)kbase.us<mailto:[email protected]>" <release-team(a)kbase.us<mailto:[email protected]>> Subject: Re: [Release-team] KBase Module status as of Feb. 3 Commitment from whom? I don’t know anything about this. On Feb 3, 2014, at 8:52 AM, Land, Miriam L. <landml(a)ornl.gov<mailto:[email protected]>> wrote: kb_seed/Central Store – we have commitment that it will be updated today based on new database structure

1 0