- Release-team - mailman3.cels.anl.gov

Re: [Release-team] KBase Module status as of Feb. 3
by Robert Olson 03 Feb '14

03 Feb '14

Commitment from whom? I don’t know anything about this. On Feb 3, 2014, at 8:52 AM, Land, Miriam L. <landml(a)ornl.gov> wrote: > kb_seed/Central Store – we have commitment that it will be updated today based on new database structure

1 0

KBase Module status as of Feb. 3
by Land, Miriam L. 03 Feb '14

03 Feb '14

Release Team, >From the spreadsheet, here is what I see as the status: Assembly – had git commits on Jan. 27 and 28 – I assume it is ready for deploy/test KBase FBA modeling – Had a commit yesterday – I assume it is ready for deploy/test gene_conservation – had commits on Jan. 31 – I assume it is ready for deploy/test genome_annotation – had commits on Jan. 31 – I assume it is ready for deploy/test Meme, inferelator, cmonkey – deployed and tested over the weekend. Ready for documentation testing Search – has been tested over the weekend and problems appear to be with configuration, not the code kb_seed/Central Store – we have commitment that it will be updated today based on new database structure Expression, Network, MAK, and bambi have asked for an extension to Tuesday sometime on the west coast. Are we prepared to act on the consequences of delayed modules without contingency plans? Tom, are there any waivers? Miriam From: <Land>, Miriam Land <landml(a)ornl.gov<mailto:[email protected]>> Date: Friday, January 31, 2014 10:23 AM To: KBase-devel <kbase-devel(a)lists.kbase.us<mailto:[email protected]>> Subject: [Kbase-devel] KBase Module status as of Jan. 31, 2014 - Deadline is near All, DROP DEAD DAY APPROACHES – All modules must be checked into git by 8am Eastern time Monday Feb. 3. Updates since yesterday – * id_map was unit tested had passed documentation review * AWE, Shock, and workspace_deluxe have had independent testing and no longer need black box testing. * Search is in deploy and test on the test VM As always, I welcome corrections and updates to the information below. ---------------------------------------------------------------------------------------------------------------------------- Modules expecting to update after Jan. 27 * Assembly - implementing an option for integration with the narratives. Expect completion by Feb. 3. * id_map * KBase Expression – needs DB connection and config updates * gene_conservation has end-of-month target * IRIS – What is in testing is already in production. There may be a new version before Feb. 3. * KB_seed update will be necessary if production access to the new database schema is required * Networks dependent on data load and has end-of-month target * Modeling will have a HUGE update by Thurs. (Jan. 30) Modules with a submitted contingency plan * Search Modules ready for production deployment * Authentication and Authorization Client * CBD and Probabilistic Annotation - will need black box testing at some point * Communities API – passed tests except for two that need to be withdrawn (Andreas and Travis) * Genotyping Service * Handle Service (on t6) - internal service - will need black box testing at some point * Interlog projection - will need black box testing at some point * m5nr - (Andreas and Travis) - also will need black box testing at some point * Workspace_deluxe – * Awe Service – internal service * Shock Service – internal service Modules ready to deploy to testing VM * genome_annotation – (Bob O. 1/30) Modules currently in testing * Search – (Matt Henderson 1/27) Modules ready for review of scripts, documentation, and release notes * none Modules needing updates to release notes * KBase Expression Services * Network service * AWE and Shock – release notes desirable but not mandatory * All modules that haven't been tested yet Modules waiting on database updates * FBA modeling - transitioning to the new workspace (Chris H.) * KBase Expression – need DB connection and config updates afterwards (confirmed 1/27) (Jason) Modules waiting on a developer * Assembly - option for the narrative will be added * Bambi, Cmonkey, inferelator, meme and possibly MAK will be ready 1/28 * Gene conservation – development continuing (end of month confirmed 1/22) (Gang Fang) - will need black box testing at some point * kb_seed – update to new database schema (Bob O. 1/27) * Networks - development continuing and data load dependency, by Jan. 31 (confirmed 1/22) (Pavel) Modules Withdrawn from the formal build process * Functional Ortholog Predictor – delay until after Feb. demo (1/23/14 Fei He) * Landing Pages (Parmvir) - Will not be part of this build cycle (S. Chan 1/23/14) * Narrative (Steve Chan) - Will not be part of this build cycle (S. Chan 1/23/14) Miriam _______________________________________________ Kbase-devel mailing list Kbase-devel(a)lists.kbase.us<mailto:[email protected]> https://lists.kbase.us/mailman/listinfo/kbase-devel

1 0

Re: [Release-team] [Kbase-devel] KBase Module status as of Jan. 31, 2014 - Deadline is near
by Yoo, Shinjae 03 Feb '14

03 Feb '14

Miriam, We may use non-production version of networks API in our plant narrative and I’ll monitor VM status before and during demo. Thanks, -SJ On Feb 3, 2014, at 9:14 AM, Land, Miriam L. <landml(a)ornl.gov> wrote: > Pavel, > > As a reminder, contingency plans were due on Jan. 27th. Only the search module submitted contingency plan. As a result, unless there is a waiver from Tom Brettin, the following will go into effect for your modules: > >> Consequences: If a module is not ready on Feb. 3, commits to git WILL NOT be deployed to production and WILL NOT be visible to the public in demos. Instead, the deployment team will start implementing the contingency plan. If no contingency plan is submitted, the developer is on their own. Time permitting, help may still be available but module owner is accountable, not the release team, if no contingency plan is submitted in advance. > > Miriam > > On Feb 3, 2014, at 8:46 AM, Pavel Novichkov <PSNovichkov(a)lbl.gov> wrote: > >> Hi Miriam, >> >> We are kind of exhausted with networks data load sprint………. two services are dependent on that: Expression and Networks. >> We just loaded data and need one more day to test and polish them… So we need one more day. >> >> Because of that I was not able able to finish BAMBI, I need just 1-2 hours… >> >> So we need to move deadline for these three services to Tuesday. >> >> Thanks, >> P. >> >> >

1 0

Re: [Release-team] [Kbase-devel] KBase Module status as of Jan. 31, 2014 - Deadline is near
by Land, Miriam L. 03 Feb '14

03 Feb '14

Pavel, Please let me correct one thing. This doesn't apply to all of your modules. Just the one not in the git repo this morning. Miriam From: <Land>, Miriam Land <landml(a)ornl.gov<mailto:[email protected]>> Date: Monday, February 3, 2014 9:14 AM To: Pavel Novichkov <PSNovichkov(a)lbl.gov<mailto:[email protected]>> Cc: "Thomas S. Brettin" <brettin(a)cels.anl.gov<mailto:[email protected]>>, Shinjae Yoo <sjyoo(a)bnl.gov<mailto:[email protected]>>, Jason Baumohl <jkbaumohl(a)lbl.gov<mailto:[email protected]>>, "release-team(a)kbase.us<mailto:[email protected]>" <release-team(a)kbase.us<mailto:[email protected]>> Subject: Re: [Kbase-devel] KBase Module status as of Jan. 31, 2014 - Deadline is near Pavel, As a reminder, contingency plans were due on Jan. 27th. Only the search module submitted contingency plan. As a result, unless there is a waiver from Tom Brettin, the following will go into effect for your modules: Consequences: If a module is not ready on Feb. 3, commits to git WILL NOT be deployed to production and WILL NOT be visible to the public in demos. Instead, the deployment team will start implementing the contingency plan. If no contingency plan is submitted, the developer is on their own. Time permitting, help may still be available but module owner is accountable, not the release team, if no contingency plan is submitted in advance. Miriam On Feb 3, 2014, at 8:46 AM, Pavel Novichkov <PSNovichkov(a)lbl.gov<mailto:[email protected]>> wrote: Hi Miriam, We are kind of exhausted with networks data load sprint………. two services are dependent on that: Expression and Networks. We just loaded data and need one more day to test and polish them… So we need one more day. Because of that I was not able able to finish BAMBI, I need just 1-2 hours… So we need to move deadline for these three services to Tuesday. Thanks, P.

1 0

Re: [Release-team] [Kbase-devel] KBase Module status as of Jan. 31, 2014 - Deadline is near
by Land, Miriam L. 03 Feb '14

03 Feb '14

Pavel, As a reminder, contingency plans were due on Jan. 27th. Only the search module submitted contingency plan. As a result, unless there is a waiver from Tom Brettin, the following will go into effect for your modules: > Consequences: If a module is not ready on Feb. 3, commits to git WILL NOT be deployed to production and WILL NOT be visible to the public in demos. Instead, the deployment team will start implementing the contingency plan. If no contingency plan is submitted, the developer is on their own. Time permitting, help may still be available but module owner is accountable, not the release team, if no contingency plan is submitted in advance. Miriam On Feb 3, 2014, at 8:46 AM, Pavel Novichkov <PSNovichkov(a)lbl.gov> wrote: > Hi Miriam, > > We are kind of exhausted with networks data load sprint………. two services are dependent on that: Expression and Networks. > We just loaded data and need one more day to test and polish them… So we need one more day. > > Because of that I was not able able to finish BAMBI, I need just 1-2 hours… > > So we need to move deadline for these three services to Tuesday. > > Thanks, > P. > >

1 0

Re: [Release-team] Services to add to or remove from Services page (http://kbase.us/developer-zone/services/)
by Land, Miriam L. 03 Feb '14

03 Feb '14

Babmi? From: Nomi Harris <nlharris(a)lbl.gov<mailto:[email protected]>> Date: Saturday, February 1, 2014 1:24 AM To: "release-team(a)kbase.us<mailto:[email protected]>" <release-team(a)kbase.us<mailto:[email protected]>> Cc: Nomi Harris <nlharris(a)lbl.gov<mailto:[email protected]>> Subject: Re: [Release-team] Services to add to or remove from Services page (http://kbase.us/developer-zone/services/) Forgot one: MEME The MEME service integrates the MEME suite of motif-based sequence analysis tools, which enable discovery of novel motifs in collections of unaligned DNA sequences, comparison of discovered motifs, and search for motif occurrences in sequence datasets. On Fri, Jan 31, 2014 at 9:51 PM, Nomi Harris <nlharris(a)lbl.gov<mailto:[email protected]>> wrote: This is my understanding of which new services need to be added to the Services web page and which are being removed (commented out, mostly, so we can restore them quickly if they are revived). New (user-facing) services in this release [note that I edited some of the descriptions that were in the Master tab because some of them were too wordy] cMonkey: The cMonkey biclustering tool takes as input gene expression data, promoter sequences, and gene-gene functional association/physical interaction data, and finds biclusters of conditionally co-regulated genes. Compression-based Distance (CBD): Compression-based distance (CBD) is a method to efficiently characterize the similarities between microbial communities via the amount of repetition or overlap in order to determine microbial community distance. Expression: This service has methods to retrieve data and metadata for expression samples, as well as methods for data comparisons. Supports microarray, RNA_seq, proteomics, and qPCR. Gene conservation: This module provides services for the evaluation of reliability of cross-species biological network projection to assess gene conservation. Inferelator: The Inferelator algorithm infers regulatory networks from gene expression data. It takes as input cMonkey biclusters, a list of regulators, and (optionally) environment meta-information that describes how the experiments were performed. Interlog projection: This service has methods to project an interlog network of species A from the query network of species B based on ortholog information. The ortholog information is precomputed by the Inparanoid algorithm. M5NR: The M5NR integrates many sequence databases into one single, searchable database. A similarity search (using BLAST or BLAT) allows the user to retrieve similarities to several databases. The M5NR API facilitates mappings between M5NR IDs (sequence md5 checksums) and annotations from databases integrated into the M5NR. Probabilistic annotation: This service computes likelihoods for annotations and reactions for a given genome. Likelihoods are computed based on similarity to genes with literature evidence and genes in subsystems. Commenting out from Services page because service is not currently working: PROM (already commented out) Regulation (regprecise) QC service Fizzy feature selection (removed at Tom Brettin's request on 12/9) I also propose to remove or comment out these two--let me know if I should keep them: Protein info service Translation service (The MO translation component of the translation service translates KBase IDs to MicrobesOnline IDs and vice-versa. These methods are used to retrieve MicrobesOnline annotations (such as orthologs, domains, and operons) for corresponding KBase objects.)

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Brettin, Thomas S. 03 Feb '14

03 Feb '14

Yes an overall rethink and plan is a good thing Sent from my Verizon Wireless 4G LTE DROID Gavin Price <gaprice(a)lbl.gov> wrote: Sent from my Cray XT30 On Feb 2, 2014 5:07 PM, "Brettin, Thomas S." <brettin(a)cels.anl.gov<mailto:[email protected]>> wrote: > > -------- > WARNING: At least one of the links in the message below goes to an IP address (e.g.10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > -------- > > We need a two step process. Dump all types in prod. Load onto test. That's not going to happen in the next couple weeks. > Seems backwards. It'll do for now. How we manage types in kbase needs an overall rethink, as we discussed. -g > > Sent from my Verizon Wireless 4G LTE DROID > > > Gavin Price <gaprice(a)lbl.gov<mailto:[email protected]>> wrote: > > You don't need to review them, and the mechanics of approving a type (or > a set of types) takes a couple seconds. I would just say that if a type > is available in the prod server it should be available in the test > server, and the prod server can act as the list of approved types. All > the test team needs to do is see if the requester owns the type in the > prod server and if so, approve it on the test server. This could > probably be scripted easily. > > > Why is type registration a risky operation? > Giving all users admin privileges on the test workspace service is what > I was saying seems dangerous. > > -g > > On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > > -------- > > WARNING: At least one of the links in the message below goes to an IP address (e.g. 10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > > -------- > > > > I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. > > > > Why is type registration a risky operation? > > > > --Dan > > > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov<mailto:[email protected]>> wrote: > >> > >> Firstly, you may want to check out > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > >> > >> These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. > >> > >> There are a few options that are possible: > >> > >> 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. > >> > >> 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it > >> > >> 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. > >> > >> -g > >> > >> > >>> On 2/2/2014 8:57 AM, Shane Canon wrote: > >>> Alexey, > >>> > >>> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has > >>> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. > >>> > >>> --Shane > >>> > >>> > >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov<mailto:[email protected]>> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: > >>>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). > >>>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. > >>>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. > >>>> 4. Module's owner releases module for general use of its type (Workspace.release_module). > >>>> > >>>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. > >>>> > >>>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. > >>>> > >>>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? > >>>> > >>>> Best, > >>>> Alexey > >>>> > >>>> > >>>> > >>>> > >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: > >>>>> I'm adding the release-team to the thread so they are in the loop. > >>>>> > >>>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production > >>>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in > >>>>> the workspace, it should populate it as part of the test. > >>>>> > >>>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? > >>>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded > >>>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded > >>>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. > >>>>> > >>>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but > >>>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in > >>>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it > >>>>> is necessary (i.e. a major change in the type compiler or an API change). > >>>>> > >>>>> --Shane > >>>>> > >>>>> > >>>>> > >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov<mailto:[email protected]>> wrote: > >>>>>> > >>>>>> Hi Shane, > >>>>>> > >>>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? > >>>>>> > >>>>>> Thank you, > >>>>>> Alexey > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: > >>>>>>> Pavel, > >>>>>>> > >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. > >>>>>>> > >>>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't > >>>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. > >>>>>>> > >>>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing > >>>>>>> against a workspace service that isn't the production one. So it is probably missing some > >>>>>>> objects that are used in the test. I would like to have a test I can run in the test environment > >>>>>>> just for completeness. But I think we are close. > >>>>>>> > >>>>>>> --Shane > >>>>>>> > >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov<mailto:[email protected]>> wrote: > >>>>>>>> > >>>>>>>> Hi Shane, > >>>>>>>> > >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> P. > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov<mailto:[email protected]>> wrote: > >>>>>>>>> > >>>>>>>>> Pavel, > >>>>>>>>> > >>>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for > >>>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. > >>>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references > >>>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. > >>>>>>>>> So that needs to be fixed. > >>>>>>>>> > >>>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. > >>>>>>>>> > >>>>>>>>> For now, I will work on bambi and inferelator. > >>>>>>>>> > >>>>>>>>> --Shane > >>>>>>>>> > >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test > >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' > >>>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' > >>>>>>>>> # run each test > >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ > >>>>>>>>> if [ $? -ne 0 ] ; then \ > >>>>>>>>> exit 1 ; \ > >>>>>>>>> fi \ > >>>>>>>>> > >>>>>>>>> 1..24 > >>>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws > >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws > >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 32. > >>>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat > su > >> n.reflect.Deleg > >>>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteA > da > >> pter.service(Coy > >>>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.griz > zl > >> y.SelectionKeyCo > >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} > >>>>>>>>> > >>>>>>>>> not ok 3 - MEME run successfuly > >>>>>>>>> # Failed test 'MEME run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 35. > >>>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws > >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws > >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 6 - MEME run successfuly > >>>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws > >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 50. > >>>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccesso > rI > >> mpl.invoke(Nati > >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. > ja > >> va:331)\n\tat or > >>>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall > (P > >> rotocolChainCont > >>>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} > >>>>>>>>> > >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly > >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 53. > >>>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws > >>>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 59. > >>>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:77 > 0) > >> \n\tat org.apac > >>>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask > .d > >> oProcess(Process > >>>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"6 > 56 > >> 565433923738"} > >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly > >>>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws > >>>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws > >>>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 69. > >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> not ok 15 - TOMTOM run successfuly > >>>>>>>>> # Failed test 'TOMTOM run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 72. > >>>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws > >>>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws > >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 18 - TOMTOM run successfuly > >>>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws > >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws > >>>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 87. > >>>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl. > in > >> voke(NativeMeth > >>>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 > 31 > >> )\n\tat org.apac > >>>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Proto > co > >> lChainContextTas > >>>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} > >>>>>>>>> > >>>>>>>>> not ok 21 - MAST run successfuly > >>>>>>>>> # Failed test 'MAST run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 90. > >>>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws > >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws > >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 24 - MAST run successfuly > >>>>>>>>> # Looks like you failed 9 tests of 24. > >>>>>>>>> make: *** [test-scripts] Error 1 > >> _______________________________________________ > >> Release-team mailing list > >> Release-team(a)lists.kbase.us<mailto:[email protected]> > >> https://lists.kbase.us/mailman/listinfo/release-team > > > > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us<mailto:[email protected]> > https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Gavin Price 02 Feb '14

02 Feb '14

Sent from my Cray XT30 On Feb 2, 2014 5:07 PM, "Brettin, Thomas S." <brettin(a)cels.anl.gov> wrote: > > -------- > WARNING: At least one of the links in the message below goes to an IP address (e.g.10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > -------- > > We need a two step process. Dump all types in prod. Load onto test. That's not going to happen in the next couple weeks. > Seems backwards. It'll do for now. How we manage types in kbase needs an overall rethink, as we discussed. -g > > Sent from my Verizon Wireless 4G LTE DROID > > > Gavin Price <gaprice(a)lbl.gov> wrote: > > You don't need to review them, and the mechanics of approving a type (or > a set of types) takes a couple seconds. I would just say that if a type > is available in the prod server it should be available in the test > server, and the prod server can act as the list of approved types. All > the test team needs to do is see if the requester owns the type in the > prod server and if so, approve it on the test server. This could > probably be scripted easily. > > > Why is type registration a risky operation? > Giving all users admin privileges on the test workspace service is what > I was saying seems dangerous. > > -g > > On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > > -------- > > WARNING: At least one of the links in the message below goes to an IP address (e.g. 10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > > -------- > > > > I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. > > > > Why is type registration a risky operation? > > > > --Dan > > > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov> wrote: > >> > >> Firstly, you may want to check out > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > >> > >> These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. > >> > >> There are a few options that are possible: > >> > >> 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. > >> > >> 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it > >> > >> 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. > >> > >> -g > >> > >> > >>> On 2/2/2014 8:57 AM, Shane Canon wrote: > >>> Alexey, > >>> > >>> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has > >>> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. > >>> > >>> --Shane > >>> > >>> > >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: > >>>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). > >>>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. > >>>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. > >>>> 4. Module's owner releases module for general use of its type (Workspace.release_module). > >>>> > >>>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. > >>>> > >>>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. > >>>> > >>>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? > >>>> > >>>> Best, > >>>> Alexey > >>>> > >>>> > >>>> > >>>> > >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: > >>>>> I'm adding the release-team to the thread so they are in the loop. > >>>>> > >>>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production > >>>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in > >>>>> the workspace, it should populate it as part of the test. > >>>>> > >>>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? > >>>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded > >>>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded > >>>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. > >>>>> > >>>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but > >>>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in > >>>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it > >>>>> is necessary (i.e. a major change in the type compiler or an API change). > >>>>> > >>>>> --Shane > >>>>> > >>>>> > >>>>> > >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: > >>>>>> > >>>>>> Hi Shane, > >>>>>> > >>>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? > >>>>>> > >>>>>> Thank you, > >>>>>> Alexey > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: > >>>>>>> Pavel, > >>>>>>> > >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. > >>>>>>> > >>>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't > >>>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. > >>>>>>> > >>>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing > >>>>>>> against a workspace service that isn't the production one. So it is probably missing some > >>>>>>> objects that are used in the test. I would like to have a test I can run in the test environment > >>>>>>> just for completeness. But I think we are close. > >>>>>>> > >>>>>>> --Shane > >>>>>>> > >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: > >>>>>>>> > >>>>>>>> Hi Shane, > >>>>>>>> > >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> P. > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: > >>>>>>>>> > >>>>>>>>> Pavel, > >>>>>>>>> > >>>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for > >>>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. > >>>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references > >>>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. > >>>>>>>>> So that needs to be fixed. > >>>>>>>>> > >>>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. > >>>>>>>>> > >>>>>>>>> For now, I will work on bambi and inferelator. > >>>>>>>>> > >>>>>>>>> --Shane > >>>>>>>>> > >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test > >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' > >>>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' > >>>>>>>>> # run each test > >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ > >>>>>>>>> if [ $? -ne 0 ] ; then \ > >>>>>>>>> exit 1 ; \ > >>>>>>>>> fi \ > >>>>>>>>> > >>>>>>>>> 1..24 > >>>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws > >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws > >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 32. > >>>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat > su > >> n.reflect.Deleg > >>>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteA > da > >> pter.service(Coy > >>>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.griz > zl > >> y.SelectionKeyCo > >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} > >>>>>>>>> > >>>>>>>>> not ok 3 - MEME run successfuly > >>>>>>>>> # Failed test 'MEME run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 35. > >>>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws > >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws > >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 6 - MEME run successfuly > >>>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws > >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 50. > >>>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccesso > rI > >> mpl.invoke(Nati > >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. > ja > >> va:331)\n\tat or > >>>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall > (P > >> rotocolChainCont > >>>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} > >>>>>>>>> > >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly > >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 53. > >>>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws > >>>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws > >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 59. > >>>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:77 > 0) > >> \n\tat org.apac > >>>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask > .d > >> oProcess(Process > >>>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"6 > 56 > >> 565433923738"} > >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly > >>>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws > >>>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws > >>>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 69. > >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> not ok 15 - TOMTOM run successfuly > >>>>>>>>> # Failed test 'TOMTOM run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 72. > >>>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws > >>>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws > >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 18 - TOMTOM run successfuly > >>>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws > >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws > >>>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' > >>>>>>>>> # at test/script_tests-command-line.t line 87. > >>>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl. > in > >> voke(NativeMeth > >>>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 > 31 > >> )\n\tat org.apac > >>>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Proto > co > >> lChainContextTas > >>>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} > >>>>>>>>> > >>>>>>>>> not ok 21 - MAST run successfuly > >>>>>>>>> # Failed test 'MAST run successfuly' > >>>>>>>>> # at test/script_tests-command-line.t line 90. > >>>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws > >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws > >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> ok 24 - MAST run successfuly > >>>>>>>>> # Looks like you failed 9 tests of 24. > >>>>>>>>> make: *** [test-scripts] Error 1 > >> _______________________________________________ > >> Release-team mailing list > >> Release-team(a)lists.kbase.us > >> https://lists.kbase.us/mailman/listinfo/release-team > > > > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Brettin, Thomas S. 02 Feb '14

02 Feb '14

We need a two step process. Dump all types in prod. Load onto test. Seems backwards. Sent from my Verizon Wireless 4G LTE DROID Gavin Price <gaprice(a)lbl.gov> wrote: You don't need to review them, and the mechanics of approving a type (or a set of types) takes a couple seconds. I would just say that if a type is available in the prod server it should be available in the test server, and the prod server can act as the list of approved types. All the test team needs to do is see if the requester owns the type in the prod server and if so, approve it on the test server. This could probably be scripted easily. > Why is type registration a risky operation? Giving all users admin privileges on the test workspace service is what I was saying seems dangerous. -g On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > -------- > WARNING: At least one of the links in the message below goes to an IP address (e.g. 10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > -------- > > I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. > > Why is type registration a risky operation? > > --Dan > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov> wrote: >> >> Firstly, you may want to check out >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… >> >> These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. >> >> There are a few options that are possible: >> >> 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. >> >> 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it >> >> 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. >> >> -g >> >> >>> On 2/2/2014 8:57 AM, Shane Canon wrote: >>> Alexey, >>> >>> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has >>> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. >>> >>> --Shane >>> >>> >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>>> >>>> Hi all, >>>> >>>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: >>>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). >>>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. >>>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. >>>> 4. Module's owner releases module for general use of its type (Workspace.release_module). >>>> >>>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. >>>> >>>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. >>>> >>>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? >>>> >>>> Best, >>>> Alexey >>>> >>>> >>>> >>>> >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: >>>>> I'm adding the release-team to the thread so they are in the loop. >>>>> >>>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production >>>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in >>>>> the workspace, it should populate it as part of the test. >>>>> >>>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? >>>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded >>>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded >>>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. >>>>> >>>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but >>>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in >>>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it >>>>> is necessary (i.e. a major change in the type compiler or an API change). >>>>> >>>>> --Shane >>>>> >>>>> >>>>> >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>>>>> >>>>>> Hi Shane, >>>>>> >>>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? >>>>>> >>>>>> Thank you, >>>>>> Alexey >>>>>> >>>>>> >>>>>> >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: >>>>>>> Pavel, >>>>>>> >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >>>>>>> >>>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >>>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >>>>>>> >>>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing >>>>>>> against a workspace service that isn't the production one. So it is probably missing some >>>>>>> objects that are used in the test. I would like to have a test I can run in the test environment >>>>>>> just for completeness. But I think we are close. >>>>>>> >>>>>>> --Shane >>>>>>> >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: >>>>>>>> >>>>>>>> Hi Shane, >>>>>>>> >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> P. >>>>>>>> >>>>>>>> >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: >>>>>>>>> >>>>>>>>> Pavel, >>>>>>>>> >>>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>>>>>>> So that needs to be fixed. >>>>>>>>> >>>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>>>>>>> >>>>>>>>> For now, I will work on bambi and inferelator. >>>>>>>>> >>>>>>>>> --Shane >>>>>>>>> >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>>>>>>> # run each test >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>>>>>>> if [ $? -ne 0 ] ; then \ >>>>>>>>> exit 1 ; \ >>>>>>>>> fi \ >>>>>>>>> >>>>>>>>> 1..24 >>>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 32. >>>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat su >> n.reflect.Deleg >>>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteA da >> pter.service(Coy >>>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.griz zl >> y.SelectionKeyCo >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>>>>>>> >>>>>>>>> not ok 3 - MEME run successfuly >>>>>>>>> # Failed test 'MEME run successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 35. >>>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 >>>>>>>>> >>>>>>>>> >>>>>>>>> ok 6 - MEME run successfuly >>>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 50. >>>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccesso rI >> mpl.invoke(Nati >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter. ja >> va:331)\n\tat or >>>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall (P >> rotocolChainCont >>>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>>>>>>> >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 53. >>>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 59. >>>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:77 0) >> \n\tat org.apac >>>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask .d >> oProcess(Process >>>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"6 56 >> 565433923738"} >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly >>>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 69. >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> not ok 15 - TOMTOM run successfuly >>>>>>>>> # Failed test 'TOMTOM run successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 72. >>>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 >>>>>>>>> >>>>>>>>> >>>>>>>>> ok 18 - TOMTOM run successfuly >>>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 87. >>>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl. in >> voke(NativeMeth >>>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:3 31 >> )\n\tat org.apac >>>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Proto co >> lChainContextTas >>>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>>>>>>> >>>>>>>>> not ok 21 - MAST run successfuly >>>>>>>>> # Failed test 'MAST run successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 90. >>>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 >>>>>>>>> >>>>>>>>> >>>>>>>>> ok 24 - MAST run successfuly >>>>>>>>> # Looks like you failed 9 tests of 24. >>>>>>>>> make: *** [test-scripts] Error 1 >> _______________________________________________ >> Release-team mailing list >> Release-team(a)lists.kbase.us >> https://lists.kbase.us/mailman/listinfo/release-team > _______________________________________________ Release-team mailing list Release-team(a)lists.kbase.us https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Gavin Price 02 Feb '14

02 Feb '14

You don't need to review them, and the mechanics of approving a type (or a set of types) takes a couple seconds. I would just say that if a type is available in the prod server it should be available in the test server, and the prod server can act as the list of approved types. All the test team needs to do is see if the requester owns the type in the prod server and if so, approve it on the test server. This could probably be scripted easily. > Why is type registration a risky operation? Giving all users admin privileges on the test workspace service is what I was saying seems dangerous. -g On 2/2/2014 12:19 PM, Murphy-Olson, Daniel E. wrote: > -------- > WARNING: At least one of the links in the message below goes to an IP address (e.g. 10.1.1.1), which could be malicious. To learn how to protect yourself, please go here: https://commons.lbl.gov/x/_591B > -------- > > I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. > > Why is type registration a risky operation? > > --Dan > >> On Feb 2, 2014, at 12:04 PM, "Gavin Price"<gaprice(a)lbl.gov> wrote: >> >> Firstly, you may want to check out >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… >> http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… >> >> These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. >> >> There are a few options that are possible: >> >> 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. >> >> 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it >> >> 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. >> >> -g >> >> >>> On 2/2/2014 8:57 AM, Shane Canon wrote: >>> Alexey, >>> >>> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has >>> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. >>> >>> --Shane >>> >>> >>>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>>> >>>> Hi all, >>>> >>>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: >>>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). >>>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. >>>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. >>>> 4. Module's owner releases module for general use of its type (Workspace.release_module). >>>> >>>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. >>>> >>>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. >>>> >>>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? >>>> >>>> Best, >>>> Alexey >>>> >>>> >>>> >>>> >>>>> On 2/1/2014 11:41 AM, Shane Canon wrote: >>>>> I'm adding the release-team to the thread so they are in the loop. >>>>> >>>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production >>>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in >>>>> the workspace, it should populate it as part of the test. >>>>> >>>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? >>>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded >>>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded >>>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. >>>>> >>>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but >>>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in >>>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it >>>>> is necessary (i.e. a major change in the type compiler or an API change). >>>>> >>>>> --Shane >>>>> >>>>> >>>>> >>>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>>>>> >>>>>> Hi Shane, >>>>>> >>>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? >>>>>> >>>>>> Thank you, >>>>>> Alexey >>>>>> >>>>>> >>>>>> >>>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: >>>>>>> Pavel, >>>>>>> >>>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >>>>>>> >>>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >>>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >>>>>>> >>>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing >>>>>>> against a workspace service that isn't the production one. So it is probably missing some >>>>>>> objects that are used in the test. I would like to have a test I can run in the test environment >>>>>>> just for completeness. But I think we are close. >>>>>>> >>>>>>> --Shane >>>>>>> >>>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: >>>>>>>> >>>>>>>> Hi Shane, >>>>>>>> >>>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> P. >>>>>>>> >>>>>>>> >>>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: >>>>>>>>> >>>>>>>>> Pavel, >>>>>>>>> >>>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>>>>>>> So that needs to be fixed. >>>>>>>>> >>>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>>>>>>> >>>>>>>>> For now, I will work on bambi and inferelator. >>>>>>>>> >>>>>>>>> --Shane >>>>>>>>> >>>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>>>>>>> # run each test >>>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>>>>>>> if [ $? -ne 0 ] ; then \ >>>>>>>>> exit 1 ; \ >>>>>>>>> fi \ >>>>>>>>> >>>>>>>>> 1..24 >>>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 32. >>>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat su >> n.reflect.Deleg >>>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAda >> pter.service(Coy >>>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzl >> y.SelectionKeyCo >>>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>>>>>>> >>>>>>>>> not ok 3 - MEME run successfuly >>>>>>>>> # Failed test 'MEME run successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 35. >>>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 >>>>>>>>> >>>>>>>>> >>>>>>>>> ok 6 - MEME run successfuly >>>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 50. >>>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorI >> mpl.invoke(Nati >>>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.ja >> va:331)\n\tat or >>>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(P >> rotocolChainCont >>>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>>>>>>> >>>>>>>>> not ok 9 - MEME PSPM collection generated successfuly >>>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 53. >>>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 59. >>>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770) >> \n\tat org.apac >>>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.d >> oProcess(Process >>>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656 >> 565433923738"} >>>>>>>>> ok 12 - MEME PSPM collection generated successfuly >>>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 69. >>>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> not ok 15 - TOMTOM run successfuly >>>>>>>>> # Failed test 'TOMTOM run successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 72. >>>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>>>> Job ID: 52ebf85be4b0b278264a5395 >>>>>>>>> >>>>>>>>> >>>>>>>>> ok 18 - TOMTOM run successfuly >>>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>>>>>>> # at test/script_tests-command-line.t line 87. >>>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.in >> voke(NativeMeth >>>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331 >> )\n\tat org.apac >>>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Protoco >> lChainContextTas >>>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>>>>>>> >>>>>>>>> not ok 21 - MAST run successfuly >>>>>>>>> # Failed test 'MAST run successfuly' >>>>>>>>> # at test/script_tests-command-line.t line 90. >>>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 >>>>>>>>> >>>>>>>>> >>>>>>>>> ok 24 - MAST run successfuly >>>>>>>>> # Looks like you failed 9 tests of 24. >>>>>>>>> make: *** [test-scripts] Error 1 >> _______________________________________________ >> Release-team mailing list >> Release-team(a)lists.kbase.us >> https://lists.kbase.us/mailman/listinfo/release-team >

1 0