Alexey, You raise some interesting issues. I hadn't appreciated some of these complexities. I'm glad you included Gavin in the chain. I'm hoping he has some ideas. For testing purposes perhaps we could upload a snapshot of the production types into the test workspace. --Shane On Feb 1, 2014, at 9:20 PM, Alexey Kazakov <[email protected]> wrote:
Hi all,
workspace service 0.1.x that we use now has several big differences from the previous workspace version. In particular, it requires registration of object types for every module to enable the objects to be saved in workspace, and only module's owner can register typespec for the module. Thus, several actions must be performed before workspace will be populated with data: 1. Module's owner requests an authorship of the module (Workspace.request_module_ownership function). 2. Workspace administrator approves the request (workspace service WILL NOT notify module's owner about it). This step prevents implementation of the whole process as a single script. 3. Module's owner registers typespec file and defines a list of types that will be available in workspace (Workspace.register_typespec function). At this step, typespec from another workspace can be copied by Workspace.register_typespec_copy function. 4. Module's owner releases module for general use of its type (Workspace.release_module).
These four steps can be performed once, as a preparation of a new workspace installation for data loading. But version numbers for all registered data types will be 1.0, and I anticipate a potential source of problems here for modules that use previous versions of data types. I do not use hard-coded version numbers in meme, cmonkey and inferelator services, but some other modules may be affected.
If execution of a module depends on other modules or their data types, their typespec files must be registered too. For instance, meme module depends on types from KBaseSequences, cmonkey depends on types from MEME, KBaseExpression, KBaseGenomes, KBaseNetworks typespecs, and inferelator depends on types from Cmonkey, KBaseGenomes and KBaseExpression typespecs.
Ideally, deployment of workspace 0.1.x in test environment should include requests of authorship for all modules currently registered in production workspace, with subsequent import of all typespecs (including all versions of typespec) in a proper order from production workspace. This will re-create all versions of data types from production workspace to ensure identical behavior of modules in test and production environments. What you think about it?
Best, Alexey
On 2/1/2014 11:41 AM, Shane Canon wrote:
I'm adding the release-team to the thread so they are in the loop.
The test workspace is a standard deployment in the test infrastructure. The only difference is it isn't populated with the data that is in the production instance. In general, it would be best if the tests could assume that there isn't any data pre-loaded into a data store. So if it requires some data in the workspace, it should populate it as part of the test.
In looking through the modules I see that there are various references to host in 140.84.*.* and 140.85.*.*. Are these "production" services? I'm worried that what we are deploying is actually contacting test services on Magellan. Also, we would prefer that URLs not be hard coded in the source code, but instead be read from the deployment configuration file. Right now it appears that all of this information is hard-coded in the java files, and we would need to recompile and redeploy if we needed to change a URL.
By the way, my workaround for the type compiler issue is that I have add the dev branch of the type compiler as a submodule. This isn't ideal but at least provides a way for us to move forward. I will try to create a branch with these changes so that you can review them and merge them in if they seem reasonable. Another option would be for you to add the generated code to the module repo and only run the type comp when it is necessary (i.e. a major change in the type compiler or an API change).
--Shane
On Jan 31, 2014, at 5:26 PM, Alexey Kazakov<[email protected]> wrote:
Hi Shane,
cmonkey and inferelator tests should pass because these services call AWE client which is doing all calculations. Tests of three functions of meme that call AWE client passed too. I am trying to figure out how to deal with other meme functions that run MEME locally. How the workspace service was configured for testing environment? Were modules and object types from production WS made available for the testing WS, or this is only a new clean installation?
Thank you, Alexey
On 1/31/2014 4:48 PM, Shane Canon wrote:
Pavel,
Thanks for pointing me to Alexey. I apologize that I keep forgetting who was working on this.
BTW, I have inferelator up and running. It seemed to pass the tests easily. There doesn't appear to be any code in bambi (just a spec file). So there isn't anything to deploy there yet.
So the only one that we need to figure out is meme. This appears to just be due to testing against a workspace service that isn't the production one. So it is probably missing some objects that are used in the test. I would like to have a test I can run in the test environment just for completeness. But I think we are close.
--Shane
On Jan 31, 2014, at 2:50 PM, Pavel Novichkov<[email protected]> wrote:
Hi Shane,
Alexey (cc-ed) was the primary developer of meme, cmnokey and inferelator services. Please talk to Alexey.
Thanks, P.
On Jan 31, 2014, at 1:01 PM, Shane Canon<[email protected]> wrote:
Pavel,
I'm working through the services. I have cmonkey and meme deployed. I've noticed a few things that we will need to fix, but I'm working around for now. Some of these are artifacts due to we no longer install into /kb/deployment. So if the code or tests assume that, then it can cause problems. We also install as a non-root user. So this can sometime lead to problems too. If you or Roman grep through the code and look for references to /kb/deployment, then you will see what needs to be addressed. I also noticed that the tests have hard coded references for a magellan host. So that needs to be fixed.
After deploying meme, I am seeing the following failures when I run make test. I could use some help to understand how to deal with these.
For now, I will work on bambi and inferelator.
--Shane
ubuntu@t46:/kbase/deployments/20140109-prod/dev_container/modules/meme$ make test ../../tools/Makefile.common.rules:36: warning: overriding commands for target `deploy-libs' Makefile:47: warning: ignoring old commands for target `deploy-libs' # run each test /kbase/runtimes/20140109-prod/bin/perl test/script_tests-command-line.t ; \ if [ $? -ne 0 ] ; then \ exit 1 ; \ fi \
1..24 ok 1 - creating Test::Cmd object for find_motifs_with_meme_from_ws not ok 2 - Running find_motifs_with_meme_from_ws # Failed test 'Running find_motifs_with_meme_from_ws' # at test/script_tests-command-line.t line 32. MemeRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeJobFromWs(MemeServerImpl.java:594)\n\tat us.kbase.meme.MemeServerImpl.findMotifsWithMemeFromWs(MemeServerImpl.java:559)\n\tat us.kbase.meme.MemeServerCaller.findMotifsWithMemeFromWs(MemeServerCaller.java:60)\n\tat us.kbase.meme.MEMEServer.findMotifsWithMemeFromWs(MEMEServer.java:103)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.Deleg atingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(Coy oteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyCo ntextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"995521465406405"}
not ok 3 - MEME run successfuly # Failed test 'MEME run successfuly' # at test/script_tests-command-line.t line 35. ok 4 - creating Test::Cmd object for find_motifs_with_meme_job_from_ws ok 5 - Running find_motifs_with_meme_job_from_ws Job ID: 52ebf83ee4b0b278264a5393
ok 6 - MEME run successfuly ok 7 - creating Test::Cmd object for get_pspm_collection_from_meme_result_from_ws not ok 8 - Running get_pspm_collection_from_meme_result_from_ws # Failed test 'Running get_pspm_collection_from_meme_result_from_ws' # at test/script_tests-command-line.t line 50. MemePSPMCollection ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:917)\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeResultFromWs(MemeServerImpl.java:882)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultFromWs(MemeServerCaller.java:488)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultFromWs(MEMEServer.java:338)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(Nati veMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat or g.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainCont extTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"633289031774389"}
not ok 9 - MEME PSPM collection generated successfuly # Failed test 'MEME PSPM collection generated successfuly' # at test/script_tests-command-line.t line 53. ok 10 - creating Test::Cmd object for get_pspm_collection_from_meme_result_job_from_ws not ok 11 - Running get_pspm_collection_from_meme_result_job_from_ws # Failed test 'Running get_pspm_collection_from_meme_result_job_from_ws' # at test/script_tests-command-line.t line 59. Job ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"MemeRunResult download error","error":"java.lang.Exception: MemeRunResult download error\n\tat us.kbase.meme.MemeServerImpl.getPspmCollectionFromMemeJobResultFromWs(MemeServerImpl.java:919)\n\tat us.kbase.meme.MemeServerCaller.getPspmCollectionFromMemeResultJobFromWs(MemeServerCaller.java:502)\n\tat us.kbase.meme.MEMEServer.getPspmCollectionFromMemeResultJobFromWs(MEMEServer.java:358)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apac he.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(Process orTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"656565433923738"}
ok 12 - MEME PSPM collection generated successfuly ok 13 - creating Test::Cmd object for compare_motifs_with_tomtom_by_collection_from_ws not ok 14 - Running compare_motifs_with_tomtom_by_collection_from_ws # Failed test 'Running compare_motifs_with_tomtom_by_collection_from_ws' # at test/script_tests-command-line.t line 69. TomtomRunResult ID: Failed to acquire token: Failed to get auth token: Failed to query Globus Online: 500 read timeout at /kbase/deployments/20140109-prod//lib/Bio/KBase/AuthToken.pm line 574.
not ok 15 - TOMTOM run successfuly # Failed test 'TOMTOM run successfuly' # at test/script_tests-command-line.t line 72. ok 16 - creating Test::Cmd object for compare_motifs_with_tomtom_job_by_collection_from_ws ok 17 - Running compare_motifs_with_tomtom_job_by_collection_from_ws Job ID: 52ebf85be4b0b278264a5395
ok 18 - TOMTOM run successfuly ok 19 - creating Test::Cmd object for find_sites_with_mast_by_collection_from_ws not ok 20 - Running find_sites_with_mast_by_collection_from_ws # Failed test 'Running find_sites_with_mast_by_collection_from_ws' # at test/script_tests-command-line.t line 87. MastRunResult ID: {"version":"1.1","error":{"name":"JSONRPCError","code":-32500,"message":"id cannot be null or the empty string","error":"us.kbase.common.service.ServerException: id cannot be null or the empty string\n\tat us.kbase.common.service.JsonClientCaller.jsonrpcCall(JsonClientCaller.java:155)\n\tat us.kbase.userandjobstate.UserAndJobStateClient.completeJob(UserAndJobStateClient.java:400)\n\tat us.kbase.meme.MemeServerImpl.finishJobWithError(MemeServerImpl.java:113)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastJobByCollectionFromWs(MemeServerImpl.java:1298)\n\tat us.kbase.meme.MemeServerImpl.findSitesWithMastByCollectionFromWs(MemeServerImpl.java:1260)\n\tat us.kbase.meme.MemeServerCaller.findSitesWithMastByCollectionFromWs(MemeServerCaller.java:287)\n\tat us.kbase.meme.MEMEServer.findSitesWithMastByCollectionFromWs(MEMEServer.java:279)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth odAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat us.kbase.common.service.JsonServerServlet.doPost(JsonServerServlet.java:303)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:688)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:770)\n\tat org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)\n\tat org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)\n\tat org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)\n\tat org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)\n\tat org.apac he.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)\n\tat com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)\n\tat com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860)\n\tat com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757)\n\tat com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056)\n\tat com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229)\n\tat com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)\n\tat com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)\n\tat com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)\n\tat com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTas k.java:54)\n\tat com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)\n\tat com.sun.grizzly.ContextTask.run(ContextTask.java:71)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)\n\tat com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)\n\tat java.lang.Thread.run(Thread.java:744)\n"},"id":"125399939805853"}
not ok 21 - MAST run successfuly # Failed test 'MAST run successfuly' # at test/script_tests-command-line.t line 90. ok 22 - creating Test::Cmd object for find_sites_with_mast_job_by_collection_from_ws ok 23 - Running find_sites_with_mast_job_by_collection_from_ws Job ID: 52ebf86ce4b0b278264a5396
ok 24 - MAST run successfuly # Looks like you failed 9 tests of 24. make: *** [test-scripts] Error 1