Release-team

Download

[email protected]

February 2014

17 participants
148 discussions

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Murphy-Olson, Daniel E. 02 Feb '14

02 Feb '14

I don't think the test team has the bandwidth to review and approve types. I wouldn't even know where to start with criteria. Why is type registration a risky operation? --Dan > On Feb 2, 2014, at 12:04 PM, "Gavin Price" <gaprice(a)lbl.gov> wrote: > > Firstly, you may want to check out > http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… > > These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. > > There are a few options that are possible: > > 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. > > 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it > > 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. > > -g > > >> On 2/2/2014 8:57 AM, Shane Canon wrote: >> Alexey, >> >> You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has >> some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. >> >> --Shane >> >> >>> On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>> >>> Hi all, >>> >>> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: >>> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). >>> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. >>> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. >>> 4. Module's owner releases module for general use of its type (Workspace.release_module). >>> >>> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. >>> >>> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. >>> >>> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? >>> >>> Best, >>> Alexey >>> >>> >>> >>> >>>> On 2/1/2014 11:41 AM, Shane Canon wrote: >>>> I'm adding the release-team to the thread so they are in the loop. >>>> >>>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production >>>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in >>>> the workspace, it should populate it as part of the test. >>>> >>>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? >>>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded >>>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded >>>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. >>>> >>>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but >>>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in >>>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it >>>> is necessary (i.e. a major change in the type compiler or an API change). >>>> >>>> --Shane >>>> >>>> >>>> >>>>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>>>> >>>>> Hi Shane, >>>>> >>>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? >>>>> >>>>> Thank you, >>>>> Alexey >>>>> >>>>> >>>>> >>>>>> On 1/31/2014 4:48 PM, Shane Canon wrote: >>>>>> Pavel, >>>>>> >>>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >>>>>> >>>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >>>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >>>>>> >>>>>> So the only one that we need to figure out is meme. This appears to just be due to testing >>>>>> against a workspace service that isn't the production one. So it is probably missing some >>>>>> objects that are used in the test. I would like to have a test I can run in the test environment >>>>>> just for completeness. But I think we are close. >>>>>> >>>>>> --Shane >>>>>> >>>>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: >>>>>>> >>>>>>> Hi Shane, >>>>>>> >>>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>>>>>> >>>>>>> Thanks, >>>>>>> P. >>>>>>> >>>>>>> >>>>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: >>>>>>>> >>>>>>>> Pavel, >>>>>>>> >>>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>>>>>> So that needs to be fixed. >>>>>>>> >>>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>>>>>> >>>>>>>> For now, I will work on bambi and inferelator. >>>>>>>> >>>>>>>> --Shane >>>>>>>> >>>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>>>>>> # run each test >>>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>>>>>> if [ $? -ne 0 ] ; then \ >>>>>>>> exit 1 ; \ >>>>>>>> fi \ >>>>>>>> >>>>>>>> 1..24 >>>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>>>>>> # at test/script_tests-command-line.t line 32. >>>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat su > n.reflect.Deleg >>>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAda > pter.service(Coy >>>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzl > y.SelectionKeyCo >>>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>>>>>> >>>>>>>> not ok 3 - MEME run successfuly >>>>>>>> # Failed test 'MEME run successfuly' >>>>>>>> # at test/script_tests-command-line.t line 35. >>>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>>>>>> Job ID: 52ebf83ee4b0b278264a5393 >>>>>>>> >>>>>>>> >>>>>>>> ok 6 - MEME run successfuly >>>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>>>>>> # at test/script_tests-command-line.t line 50. >>>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorI > mpl.invoke(Nati >>>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.ja > va:331)\n\tat or >>>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(P > rotocolChainCont >>>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>>>>>> >>>>>>>> not ok 9 - MEME PSPM collection generated successfuly >>>>>>>> # Failed test 'MEME PSPM collection generated successfuly' >>>>>>>> # at test/script_tests-command-line.t line 53. >>>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>>>>>> # at test/script_tests-command-line.t line 59. >>>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770) > \n\tat org.apac >>>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.d > oProcess(Process >>>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656 > 565433923738"} >>>>>>>> >>>>>>>> ok 12 - MEME PSPM collection generated successfuly >>>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>>>>>> # at test/script_tests-command-line.t line 69. >>>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> not ok 15 - TOMTOM run successfuly >>>>>>>> # Failed test 'TOMTOM run successfuly' >>>>>>>> # at test/script_tests-command-line.t line 72. >>>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>>> Job ID: 52ebf85be4b0b278264a5395 >>>>>>>> >>>>>>>> >>>>>>>> ok 18 - TOMTOM run successfuly >>>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>>>>>> # at test/script_tests-command-line.t line 87. >>>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.in > voke(NativeMeth >>>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331 > )\n\tat org.apac >>>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(Protoco > lChainContextTas >>>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>>>>>> >>>>>>>> not ok 21 - MAST run successfuly >>>>>>>> # Failed test 'MAST run successfuly' >>>>>>>> # at test/script_tests-command-line.t line 90. >>>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>>>>>> Job ID: 52ebf86ce4b0b278264a5396 >>>>>>>> >>>>>>>> >>>>>>>> ok 24 - MAST run successfuly >>>>>>>> # Looks like you failed 9 tests of 24. >>>>>>>> make: *** [test-scripts] Error 1 > > _______________________________________________ > Release-team mailing list > Release-team(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/release-team

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Gavin Price 02 Feb '14

02 Feb '14

Firstly, you may want to check out http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… http://140.221.84.209/javadoc/us/kbase/workspace/WorkspaceClient.html#trans… These will allow you to translate type names between services even if the types were not loaded in the same order. Generally I'd recommend services use the MD5 style type in their code/settings and translate them to human readable types on startup. There are a few options that are possible: 1) The test team works with each module author to get the appropriate types installed in the test workspace. That means the module author needs to request the module name on the test server and a test team rep needs to approve that request via the admin interface. Then the module author is free to upload types as necessary. 2) The test team sets up a ws admin account and provides the credentials to testers and module authors, who can then approve their own modules. This seems a bit dangerous and I probably wouldn't recommend it 3) The test team owns all the modules in the test ws and installs any necessary types for testing themselves. -g On 2/2/2014 8:57 AM, Shane Canon wrote: > Alexey, > > You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has > some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. > > --Shane > > > On Feb 1, 2014, at 9:20 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: > >> Hi all, >> >> workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: >> 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). >> 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. >> 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. >> 4. Module's owner releases module for general use of its type (Workspace.release_module). >> >> These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. >> >> If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. >> >> Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? >> >> Best, >> Alexey >> >> >> >> >> On 2/1/2014 11:41 AM, Shane Canon wrote: >>> I'm adding the release-team to the thread so they are in the loop. >>> >>> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production >>> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in >>> the workspace, it should populate it as part of the test. >>> >>> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? >>> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded >>> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded >>> in the java files, and we would need to recompile and redeploy if we needed to change a URL. >>> >>> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but >>> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in >>> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it >>> is necessary (i.e. a major change in the type compiler or an API change). >>> >>> --Shane >>> >>> >>> >>> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >>> >>>> Hi Shane, >>>> >>>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? >>>> >>>> Thank you, >>>> Alexey >>>> >>>> >>>> >>>> On 1/31/2014 4:48 PM, Shane Canon wrote: >>>>> Pavel, >>>>> >>>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >>>>> >>>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >>>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >>>>> >>>>> So the only one that we need to figure out is meme. This appears to just be due to testing >>>>> against a workspace service that isn't the production one. So it is probably missing some >>>>> objects that are used in the test. I would like to have a test I can run in the test environment >>>>> just for completeness. But I think we are close. >>>>> >>>>> --Shane >>>>> >>>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: >>>>> >>>>>> Hi Shane, >>>>>> >>>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>>>>> >>>>>> Thanks, >>>>>> P. >>>>>> >>>>>> >>>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: >>>>>> >>>>>>> Pavel, >>>>>>> >>>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>>>>> So that needs to be fixed. >>>>>>> >>>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>>>>> >>>>>>> For now, I will work on bambi and inferelator. >>>>>>> >>>>>>> --Shane >>>>>>> >>>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>>>>> # run each test >>>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>>>>> if [ $? -ne 0 ] ; then \ >>>>>>> exit 1 ; \ >>>>>>> fi \ >>>>>>> >>>>>>> 1..24 >>>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>>>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>>>>> # at test/script_tests-command-line.t line 32. >>>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.Deleg >>>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(Coy >>>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyCo >>>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>>>>> >>>>>>> not ok 3 - MEME run successfuly >>>>>>> # Failed test 'MEME run successfuly' >>>>>>> # at test/script_tests-command-line.t line 35. >>>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>>>>> Job ID: 52ebf83ee4b0b278264a5393 >>>>>>> >>>>>>> >>>>>>> ok 6 - MEME run successfuly >>>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>>>>> # at test/script_tests-command-line.t line 50. >>>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(Nati >>>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat or >>>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainCont >>>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>>>>> >>>>>>> not ok 9 - MEME PSPM collection generated successfuly >>>>>>> # Failed test 'MEME PSPM collection generated successfuly' >>>>>>> # at test/script_tests-command-line.t line 53. >>>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>>>>> # at test/script_tests-command-line.t line 59. >>>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apac >>>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(Process >>>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656565433923738"} >>>>>>> >>>>>>> ok 12 - MEME PSPM collection generated successfuly >>>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>>>>> # at test/script_tests-command-line.t line 69. >>>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>>>>> >>>>>>> >>>>>>> >>>>>>> not ok 15 - TOMTOM run successfuly >>>>>>> # Failed test 'TOMTOM run successfuly' >>>>>>> # at test/script_tests-command-line.t line 72. >>>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>>> Job ID: 52ebf85be4b0b278264a5395 >>>>>>> >>>>>>> >>>>>>> ok 18 - TOMTOM run successfuly >>>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>>>>> # at test/script_tests-command-line.t line 87. >>>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth >>>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apac >>>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTas >>>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>>>>> >>>>>>> not ok 21 - MAST run successfuly >>>>>>> # Failed test 'MAST run successfuly' >>>>>>> # at test/script_tests-command-line.t line 90. >>>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>>>>> Job ID: 52ebf86ce4b0b278264a5396 >>>>>>> >>>>>>> >>>>>>> ok 24 - MAST run successfuly >>>>>>> # Looks like you failed 9 tests of 24. >>>>>>> make: *** [test-scripts] Error 1 >>>>>>> >> >

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Shane Canon 02 Feb '14

02 Feb '14

Alexey, You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. --Shane On Feb 1, 2014, at 9:20 PM, Alexey Kazakov <aekazakov(a)lbl.gov> wrote: > Hi all, > > workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: > 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). > 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. > 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. > 4. Module's owner releases module for general use of its type (Workspace.release_module). > > These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. > > If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. > > Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? > > Best, > Alexey > > > > > On 2/1/2014 11:41 AM, Shane Canon wrote: >> I'm adding the release-team to the thread so they are in the loop. >> >> The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production >> instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in >> the workspace, it should populate it as part of the test. >> >> In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? >> I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded >> in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded >> in the java files, and we would need to recompile and redeploy if we needed to change a URL. >> >> By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but >> at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in >> if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it >> is necessary (i.e. a major change in the type compiler or an API change). >> >> --Shane >> >> >> >> On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: >> >>> Hi Shane, >>> >>> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? >>> >>> Thank you, >>> Alexey >>> >>> >>> >>> On 1/31/2014 4:48 PM, Shane Canon wrote: >>>> Pavel, >>>> >>>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >>>> >>>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >>>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >>>> >>>> So the only one that we need to figure out is meme. This appears to just be due to testing >>>> against a workspace service that isn't the production one. So it is probably missing some >>>> objects that are used in the test. I would like to have a test I can run in the test environment >>>> just for completeness. But I think we are close. >>>> >>>> --Shane >>>> >>>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: >>>> >>>>> Hi Shane, >>>>> >>>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>>>> >>>>> Thanks, >>>>> P. >>>>> >>>>> >>>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: >>>>> >>>>>> Pavel, >>>>>> >>>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>>>> So that needs to be fixed. >>>>>> >>>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>>>> >>>>>> For now, I will work on bambi and inferelator. >>>>>> >>>>>> --Shane >>>>>> >>>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>>>> # run each test >>>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>>>> if [ $? -ne 0 ] ; then \ >>>>>> exit 1 ; \ >>>>>> fi \ >>>>>> >>>>>> 1..24 >>>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>>>> # at test/script_tests-command-line.t line 32. >>>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.Deleg >>>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(Coy >>>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyCo >>>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>>>> >>>>>> not ok 3 - MEME run successfuly >>>>>> # Failed test 'MEME run successfuly' >>>>>> # at test/script_tests-command-line.t line 35. >>>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>>>> Job ID: 52ebf83ee4b0b278264a5393 >>>>>> >>>>>> >>>>>> ok 6 - MEME run successfuly >>>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>>>> # at test/script_tests-command-line.t line 50. >>>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(Nati >>>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat or >>>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainCont >>>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>>>> >>>>>> not ok 9 - MEME PSPM collection generated successfuly >>>>>> # Failed test 'MEME PSPM collection generated successfuly' >>>>>> # at test/script_tests-command-line.t line 53. >>>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>>>> # at test/script_tests-command-line.t line 59. >>>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apac >>>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(Process >>>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656565433923738"} >>>>>> >>>>>> ok 12 - MEME PSPM collection generated successfuly >>>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>>>> # at test/script_tests-command-line.t line 69. >>>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>>>> >>>>>> >>>>>> >>>>>> not ok 15 - TOMTOM run successfuly >>>>>> # Failed test 'TOMTOM run successfuly' >>>>>> # at test/script_tests-command-line.t line 72. >>>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>>>> Job ID: 52ebf85be4b0b278264a5395 >>>>>> >>>>>> >>>>>> ok 18 - TOMTOM run successfuly >>>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>>>> # at test/script_tests-command-line.t line 87. >>>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth >>>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apac >>>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTas >>>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>>>> >>>>>> not ok 21 - MAST run successfuly >>>>>> # Failed test 'MAST run successfuly' >>>>>> # at test/script_tests-command-line.t line 90. >>>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>>>> Job ID: 52ebf86ce4b0b278264a5396 >>>>>> >>>>>> >>>>>> ok 24 - MAST run successfuly >>>>>> # Looks like you failed 9 tests of 24. >>>>>> make: *** [test-scripts] Error 1 >>>>>> > >

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Alexey Kazakov 01 Feb '14

01 Feb '14

Hi all, workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. 4. Module's owner releases module for general use of its type (Workspace.release_module). These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected. If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs. Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it? Best, Alexey On 2/1/2014 11:41 AM, Shane Canon wrote: > I'm adding the release-team to the thread so they are in the loop. > > The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production > instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in > the workspace, it should populate it as part of the test. > > In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? > I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded > in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded > in the java files, and we would need to recompile and redeploy if we needed to change a URL. > > By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but > at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in > if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it > is necessary (i.e. a major change in the type compiler or an API change). > > --Shane > > > > On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<aekazakov(a)lbl.gov> wrote: > >> Hi Shane, >> >> cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? >> >> Thank you, >> Alexey >> >> >> >> On 1/31/2014 4:48 PM, Shane Canon wrote: >>> Pavel, >>> >>> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >>> >>> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >>> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >>> >>> So the only one that we need to figure out is meme. This appears to just be due to testing >>> against a workspace service that isn't the production one. So it is probably missing some >>> objects that are used in the test. I would like to have a test I can run in the test environment >>> just for completeness. But I think we are close. >>> >>> --Shane >>> >>> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<psnovichkov(a)lbl.gov> wrote: >>> >>>> Hi Shane, >>>> >>>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>>> >>>> Thanks, >>>> P. >>>> >>>> >>>> On Jan 31, 2014, at 1:01 PM, Shane Canon<scanon(a)lbl.gov> wrote: >>>> >>>>> Pavel, >>>>> >>>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>>> So that needs to be fixed. >>>>> >>>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>>> >>>>> For now, I will work on bambi and inferelator. >>>>> >>>>> --Shane >>>>> >>>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>>> # run each test >>>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>>> if [ $? -ne 0 ] ; then \ >>>>> exit 1 ; \ >>>>> fi \ >>>>> >>>>> 1..24 >>>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>>> # at test/script_tests-command-line.t line 32. >>>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.Deleg >>>>> atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(Coy >>>>> oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyCo >>>>> ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>>> >>>>> not ok 3 - MEME run successfuly >>>>> # Failed test 'MEME run successfuly' >>>>> # at test/script_tests-command-line.t line 35. >>>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>>> Job ID: 52ebf83ee4b0b278264a5393 >>>>> >>>>> >>>>> ok 6 - MEME run successfuly >>>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>>> # at test/script_tests-command-line.t line 50. >>>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(Nati >>>>> veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat or >>>>> g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainCont >>>>> extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>>> >>>>> not ok 9 - MEME PSPM collection generated successfuly >>>>> # Failed test 'MEME PSPM collection generated successfuly' >>>>> # at test/script_tests-command-line.t line 53. >>>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>>> # at test/script_tests-command-line.t line 59. >>>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apac >>>>> he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(Process >>>>> orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656565433923738"} >>>>> >>>>> ok 12 - MEME PSPM collection generated successfuly >>>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>>> # at test/script_tests-command-line.t line 69. >>>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>>> >>>>> >>>>> >>>>> not ok 15 - TOMTOM run successfuly >>>>> # Failed test 'TOMTOM run successfuly' >>>>> # at test/script_tests-command-line.t line 72. >>>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>>> Job ID: 52ebf85be4b0b278264a5395 >>>>> >>>>> >>>>> ok 18 - TOMTOM run successfuly >>>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>>> # at test/script_tests-command-line.t line 87. >>>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth >>>>> odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apac >>>>> he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTas >>>>> k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>>> >>>>> not ok 21 - MAST run successfuly >>>>> # Failed test 'MAST run successfuly' >>>>> # at test/script_tests-command-line.t line 90. >>>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>>> Job ID: 52ebf86ce4b0b278264a5396 >>>>> >>>>> >>>>> ok 24 - MAST run successfuly >>>>> # Looks like you failed 9 tests of 24. >>>>> make: *** [test-scripts] Error 1 >>>>>

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Shane Canon 01 Feb '14

01 Feb '14

I'm adding the release-team to the thread so they are in the loop. The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in the workspace, it should populate it as part of the test. In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded in the java files, and we would need to recompile and redeploy if we needed to change a URL. By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it is necessary (i.e. a major change in the type compiler or an API change). --Shane On Jan 31, 2014, at 5:26 PM, Alexey Kazakov <aekazakov(a)lbl.gov> wrote: > Hi Shane, > > cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation? > > Thank you, > Alexey > > > > On 1/31/2014 4:48 PM, Shane Canon wrote: >> Pavel, >> >> Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this. >> >> BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't >> appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet. >> >> So the only one that we need to figure out is meme. This appears to just be due to testing >> against a workspace service that isn't the production one. So it is probably missing some >> objects that are used in the test. I would like to have a test I can run in the test environment >> just for completeness. But I think we are close. >> >> --Shane >> >> On Jan 31, 2014, at 2:50 PM, Pavel Novichkov <psnovichkov(a)lbl.gov> wrote: >> >>> Hi Shane, >>> >>> Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. >>> >>> Thanks, >>> P. >>> >>> >>> On Jan 31, 2014, at 1:01 PM, Shane Canon <scanon(a)lbl.gov> wrote: >>> >>>> Pavel, >>>> >>>> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >>>> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >>>> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >>>> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >>>> So that needs to be fixed. >>>> >>>> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >>>> >>>> For now, I will work on bambi and inferelator. >>>> >>>> --Shane >>>> >>>> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >>>> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >>>> Makefile:47: warning: ignoring old commands for target `deploy-libs' >>>> # run each test >>>> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >>>> if [ $? -ne 0 ] ; then \ >>>> exit 1 ; \ >>>> fi \ >>>> >>>> 1..24 >>>> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >>>> not ok 2 - Running find_motifs_with_meme_from_ws >>>> # Failed test 'Running find_motifs_with_meme_from_ws' >>>> # at test/script_tests-command-line.t line 32. >>>> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >>>> >>>> not ok 3 - MEME run successfuly >>>> # Failed test 'MEME run successfuly' >>>> # at test/script_tests-command-line.t line 35. >>>> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >>>> ok 5 - Running find_motifs_with_meme_job_from_ws >>>> Job ID: 52ebf83ee4b0b278264a5393 >>>> >>>> >>>> ok 6 - MEME run successfuly >>>> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >>>> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >>>> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >>>> # at test/script_tests-command-line.t line 50. >>>> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >>>> >>>> not ok 9 - MEME PSPM collection generated successfuly >>>> # Failed test 'MEME PSPM collection generated successfuly' >>>> # at test/script_tests-command-line.t line 53. >>>> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >>>> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >>>> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >>>> # at test/script_tests-command-line.t line 59. >>>> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656565433923738"} >>>> >>>> ok 12 - MEME PSPM collection generated successfuly >>>> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >>>> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >>>> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >>>> # at test/script_tests-command-line.t line 69. >>>> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >>>> >>>> >>>> >>>> not ok 15 - TOMTOM run successfuly >>>> # Failed test 'TOMTOM run successfuly' >>>> # at test/script_tests-command-line.t line 72. >>>> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >>>> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >>>> Job ID: 52ebf85be4b0b278264a5395 >>>> >>>> >>>> ok 18 - TOMTOM run successfuly >>>> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >>>> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >>>> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >>>> # at test/script_tests-command-line.t line 87. >>>> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >>>> >>>> not ok 21 - MAST run successfuly >>>> # Failed test 'MAST run successfuly' >>>> # at test/script_tests-command-line.t line 90. >>>> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >>>> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >>>> Job ID: 52ebf86ce4b0b278264a5396 >>>> >>>> >>>> ok 24 - MAST run successfuly >>>> # Looks like you failed 9 tests of 24. >>>> make: *** [test-scripts] Error 1 >>>> >> > >

1 0

Re: [Release-team] Services to add to or remove from Services page (http://kbase.us/developer-zone/services/)
by Nomi Harris 01 Feb '14

01 Feb '14

Forgot one: MEME The MEME service integrates the MEME suite of motif-based sequence analysis tools, which enable discovery of novel motifs in collections of unaligned DNA sequences, comparison of discovered motifs, and search for motif occurrences in sequence datasets. On Fri, Jan 31, 2014 at 9:51 PM, Nomi Harris <nlharris(a)lbl.gov> wrote: > This is my understanding of which new services need to be added to the > Services web page and which are being removed (commented out, mostly, so we > can restore them quickly if they are revived). > > New (user-facing) services in this release [note that I edited some of the > descriptions that were in the Master tab because some of them were too > wordy] > > cMonkey: > The cMonkey biclustering tool takes as input gene expression data, > promoter sequences, and gene-gene functional association/physical > interaction data, and finds biclusters of conditionally co-regulated genes. > > Compression-based Distance (CBD): > Compression-based distance (CBD) is a method to efficiently characterize > the similarities between microbial communities via the amount of repetition > or overlap in order to determine microbial community distance. > > Expression: > This service has methods to retrieve data and metadata for expression > samples, as well as methods for data comparisons. Supports microarray, > RNA_seq, proteomics, and qPCR. > > Gene conservation: > This module provides services for the evaluation of reliability of > cross-species biological network projection to assess gene conservation. > > Inferelator: > The Inferelator algorithm infers regulatory networks from gene expression > data. It takes as input cMonkey biclusters, a list of regulators, and > (optionally) environment meta-information that describes how the > experiments were performed. > > Interlog projection: > This service has methods to project an interlog network of species A from > the query network of species B based on ortholog information. The ortholog > information is precomputed by the Inparanoid algorithm. > > M5NR: > The M5NR integrates many sequence databases into one single, searchable > database. A similarity search (using BLAST or BLAT) allows the user to > retrieve similarities to several databases. The M5NR API facilitates > mappings between M5NR IDs (sequence md5 checksums) and annotations from > databases integrated into the M5NR. > > Probabilistic annotation: > This service computes likelihoods for annotations and reactions for a > given genome. Likelihoods are computed based on similarity to genes with > literature evidence and genes in subsystems. > > > Commenting out from Services page because service is not currently working: > PROM (already commented out) > Regulation (regprecise) > QC service > Fizzy feature selection (removed at Tom Brettin's request on 12/9) > > I also propose to remove or comment out these two--let me know if I should > keep them: > Protein info service > Translation service (The MO translation component of the translation > service translates KBase IDs to MicrobesOnline IDs and vice-versa. These > methods are used to retrieve MicrobesOnline annotations (such as orthologs, > domains, and operons) for corresponding KBase objects.) > >

1 0

Services to add to or remove from Services page (http://kbase.us/developer-zone/services/)
by Nomi Harris 31 Jan '14

31 Jan '14

This is my understanding of which new services need to be added to the Services web page and which are being removed (commented out, mostly, so we can restore them quickly if they are revived). New (user-facing) services in this release [note that I edited some of the descriptions that were in the Master tab because some of them were too wordy] cMonkey: The cMonkey biclustering tool takes as input gene expression data, promoter sequences, and gene-gene functional association/physical interaction data, and finds biclusters of conditionally co-regulated genes. Compression-based Distance (CBD): Compression-based distance (CBD) is a method to efficiently characterize the similarities between microbial communities via the amount of repetition or overlap in order to determine microbial community distance. Expression: This service has methods to retrieve data and metadata for expression samples, as well as methods for data comparisons. Supports microarray, RNA_seq, proteomics, and qPCR. Gene conservation: This module provides services for the evaluation of reliability of cross-species biological network projection to assess gene conservation. Inferelator: The Inferelator algorithm infers regulatory networks from gene expression data. It takes as input cMonkey biclusters, a list of regulators, and (optionally) environment meta-information that describes how the experiments were performed. Interlog projection: This service has methods to project an interlog network of species A from the query network of species B based on ortholog information. The ortholog information is precomputed by the Inparanoid algorithm. M5NR: The M5NR integrates many sequence databases into one single, searchable database. A similarity search (using BLAST or BLAT) allows the user to retrieve similarities to several databases. The M5NR API facilitates mappings between M5NR IDs (sequence md5 checksums) and annotations from databases integrated into the M5NR. Probabilistic annotation: This service computes likelihoods for annotations and reactions for a given genome. Likelihoods are computed based on similarity to genes with literature evidence and genes in subsystems. Commenting out from Services page because service is not currently working: PROM (already commented out) Regulation (regprecise) QC service Fizzy feature selection (removed at Tom Brettin's request on 12/9) I also propose to remove or comment out these two--let me know if I should keep them: Protein info service Translation service (The MO translation component of the translation service translates KBase IDs to MicrobesOnline IDs and vice-versa. These methods are used to retrieve MicrobesOnline annotations (such as orthologs, domains, and operons) for corresponding KBase objects.)

1 0

Re: [Release-team] Deploying meme, cmonkey, bambi and inferelator
by Alexey Kazakov 31 Jan '14

31 Jan '14

Hi Shane, I changed service URLs in meme and cmonkey tests to http://localhost:service_port_number (assuming that tests will be started on the host where services were deployed). Test logs suggest that meme service failed to read test input from production workspace service at kbase.us/services/ws. Could you please check the access to WS service from the deployment environment? Thank you, Alexey On 1/31/2014 2:50 PM, Pavel Novichkov wrote: > Hi Shane, > > Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey. > > Thanks, > P. > > > On Jan 31, 2014, at 1:01 PM, Shane Canon <scanon(a)lbl.gov> wrote: > >> Pavel, >> >> I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for >> now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. >> We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references >> to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. >> So that needs to be fixed. >> >> After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these. >> >> For now, I will work on bambi and inferelator. >> >> --Shane >> >> ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test >> ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' >> Makefile:47: warning: ignoring old commands for target `deploy-libs' >> # run each test >> /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ >> if [ $? -ne 0 ] ; then \ >> exit 1 ; \ >> fi \ >> >> 1..24 >> ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws >> not ok 2 - Running find_motifs_with_meme_from_ws >> # Failed test 'Running find_motifs_with_meme_from_ws' >> # at test/script_tests-command-line.t line 32. >> MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"} >> >> not ok 3 - MEME run successfuly >> # Failed test 'MEME run successfuly' >> # at test/script_tests-command-line.t line 35. >> ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws >> ok 5 - Running find_motifs_with_meme_job_from_ws >> Job ID: 52ebf83ee4b0b278264a5393 >> >> >> ok 6 - MEME run successfuly >> ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws >> not ok 8 - Running get_pspm_collection_from_meme_result_from_ws >> # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' >> # at test/script_tests-command-line.t line 50. >> MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"} >> >> not ok 9 - MEME PSPM collection generated successfuly >> # Failed test 'MEME PSPM collection generated successfuly' >> # at test/script_tests-command-line.t line 53. >> ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws >> not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws >> # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' >> # at test/script_tests-command-line.t line 59. >> Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656565433923738"} >> >> ok 12 - MEME PSPM collection generated successfuly >> ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws >> not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws >> # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' >> # at test/script_tests-command-line.t line 69. >> TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574. >> >> >> >> not ok 15 - TOMTOM run successfuly >> # Failed test 'TOMTOM run successfuly' >> # at test/script_tests-command-line.t line 72. >> ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws >> ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws >> Job ID: 52ebf85be4b0b278264a5395 >> >> >> ok 18 - TOMTOM run successfuly >> ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws >> not ok 20 - Running find_sites_with_mast_by_collection_from_ws >> # Failed test 'Running find_sites_with_mast_by_collection_from_ws' >> # at test/script_tests-command-line.t line 87. >> MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"} >> >> not ok 21 - MAST run successfuly >> # Failed test 'MAST run successfuly' >> # at test/script_tests-command-line.t line 90. >> ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws >> ok 23 - Running find_sites_with_mast_job_by_collection_from_ws >> Job ID: 52ebf86ce4b0b278264a5396 >> >> >> ok 24 - MAST run successfuly >> # Looks like you failed 9 tests of 24. >> make: *** [test-scripts] Error 1 >> >

1 0

Jump to page: