March 2013 - Kbase-microcomm - mailman3.cels.anl.gov

call cancelled
by Dylan Chivian 21 Mar '13

21 Mar '13

no point. thanks, - dylan

1 0

Re: [Kbase-MicroComm] [Kbase-devel] Starting a discussion of metadata in KBase
by Stephen Chan 14 Mar '13

14 Mar '13

Folker, Thanks for the reply - I am focused on a particular deliverable, so my comments are framed in that context as a use case: On Thu, Mar 14, 2013 at 11:01 AM, Folker Meyer <folker(a)anl.gov> wrote: > - MetaZen is indeed a frontend to create spreadsheets (and prefill some fields) > - The "schema" we devised is indeed implemented in a database allowing the queries. I use "schema" because it is a very lightweight solution more an overlay really. A closer reading of the doc answered this for me, thanks for indulging my short attention span. Since the metadata solution is a lightweight overlay, and the metadata requirements seem to be driven by "meta-metadata" templates, it sounds like we could (relatively) easily use this as a metadata layer for other services. Is it being used for anything else currently, for example, to store the metadata for the Shock service? > - regarding Stephen's question about a single integrated query capability: the "schema" we use links IDs to metadata. I believe we need the ability to link all KBase objects (from narrative to metagenome reads etc.) to metadata. With many different lightweight "schemas" used along the way. If you look at the way the "schemata" are implemented, you will see they are suited for this. I'd say a single KBase ID needs to link to the only one object irrespective of location in KBase. I think that a scheme for kbase wide object ids, and common metadata layer would be a vital first step to federating the data store silos. > I think one key question we have to ask ourselves is: Do we want to work with the international standards community (e.g. GSC) and expand on their "templates" or go our own way? > > With MG-RAST we have chosen to work with the community (and are now part of this community), evolving standards and CVs where needed (sometimes too slow). For us that has produced the situation where users are happy to provide minimal data and many are asking for more metadata. We basically get the same number of complaints about too little and too much metadata --> must mean we did the right thing. ;-) > Metadata is comes out of how the data is used - and we are breaking some new ground in how the data is used (do the current communities have metadata that captures what the narrative interface use cases?) Steve

1 0

Re: [Kbase-MicroComm] [Kbase-devel] Starting a discussion of metadata in KBase
by Shiran Pasternak 14 Mar '13

14 Mar '13

On Mar 14, 2013, at 2:01 PM, Folker Meyer <folker(a)anl.gov> wrote: > - Many users find metadata annoying and don't like to provide it. Our approach has been to tell them they get better service if they provide the metadata and also provide tools that make minimal metadata easy --> Metazen. > - Marcin is right that richer metadata is important (this is solved via the "environmental packages") and the notion of different levels of metadata. Richness and ease of use are somewhat opposite goals, Doreen points out that we will have to coerce our users to do the right thing. One way might be to have tools in KBase that require certain sets of metadata to work e.g. the use of certain CVs for experimental conditions. > - We probably should go for multiple levels of metadata completeness, encouraging the users to add as much as they can, while requiring a minimal subset. At least that is what we did in MG-RAST. Hope I'm not going too off-topic here. Sunita and I have been brainstorming just a few days ago about building functionality for rudimentary annotation of ontology terms from cited literature, which is very often (always?) associated with GEO data sets. This service would essentially use natural language processing[1] to mine papers for these mappings. This would be in the very common case that GEO data is not already associated with metadata. An NLP approach, as far as current pipelines go, have very high false-positive rates. But we can model it in such way that captures a measure of confidence about the mapping, so unreliable mappings aren't taken as Truth. Maybe these annotations can serve as a very liberal baseline for categorizing entities, when users don't care about providing the mappings themselves. Shiran [1] Disclaimer: I'm NOT an NLP expert! :)

1 0

Re: [Kbase-MicroComm] [Kbase-devel] Starting a discussion of metadata in KBase
by Folker Meyer 14 Mar '13

14 Mar '13

Hi Stephen and Marcin and Doreen, good discussion so far. thanks for your comments. here are a couple of additional comments/explanations or more food for thought: - MetaZen is indeed a frontend to create spreadsheets (and prefill some fields) - The "schema" we devised is indeed implemented in a database allowing the queries. I use "schema" because it is a very lightweight solution more an overlay really. - Before we ingest the metadata files, we also validate them. Requiring the user to provide at least the minimal data in correct form and with the correct CVs - The main problem we are trying to solve is allowing hundreds of users per month to submit data accompanied by metadata into MG-RAST - We don't have the manpower to manually scrutinized the data sets submitted by users, and I don't think that is a game we want to get into in KBase. So we need to provide automated solutions with automated validation where possible. - I believe the use case that we are aiming for with KBase is to be the primary location that users upload their data to for analysis, while we will continue to automatically ingest data from third parties. I believe in the long run these (e.g. GEO) will be a special case (IFF we are successful), there is simply too much new data to worry about. Legacy data is not what we should spend too much time worrying about, in 2-3 years there will be thousands of data sets of each given type. - Many users find metadata annoying and don't like to provide it. Our approach has been to tell them they get better service if they provide the metadata and also provide tools that make minimal metadata easy --> Metazen. - Marcin is right that richer metadata is important (this is solved via the "environmental packages") and the notion of different levels of metadata. Richness and ease of use are somewhat opposite goals, Doreen points out that we will have to coerce our users to do the right thing. One way might be to have tools in KBase that require certain sets of metadata to work e.g. the use of certain CVs for experimental conditions. - We probably should go for multiple levels of metadata completeness, encouraging the users to add as much as they can, while requiring a minimal subset. At least that is what we did in MG-RAST. - regarding Stephen's question about a single integrated query capability: the "schema" we use links IDs to metadata. I believe we need the ability to link all KBase objects (from narrative to metagenome reads etc.) to metadata. With many different lightweight "schemas" used along the way. If you look at the way the "schemata" are implemented, you will see they are suited for this. I'd say a single KBase ID needs to link to the only one object irrespective of location in KBase. - in my mind the search integration is where the metadata would be queried most of the time, but I can also see lots of tools that would benefit from CVs etc. Data discovery is just one area were we are a little bit stuck w/o metadata currently I think one key question we have to ask ourselves is: Do we want to work with the international standards community (e.g. GSC) and expand on their "templates" or go our own way? With MG-RAST we have chosen to work with the community (and are now part of this community), evolving standards and CVs where needed (sometimes too slow). For us that has produced the situation where users are happy to provide minimal data and many are asking for more metadata. We basically get the same number of complaints about too little and too much metadata --> must mean we did the right thing. ;-) BTW: One important aspect of our solution is that we can capture arbitrary metadata, it is just not displayed in the controlled subset. It can all be queried in the database. Let me what you think, this is just my $0.02 Best, Folker PS: There is a new emerging standard for Biolog type experiments that is just now being established. We (as KBase) might want to get involved in this? On Mar 14, 2013, at 11:58 AM, Stephen Chan <sychan(a)lbl.gov> wrote: > Hi Folker, > Thanks for sending this out - a quick read of the document > indicates that Metazen is a tool to generate a metadata spreadsheet > that is shipped around with the main datafiles. It seems to be a good > start at a standalone app for that purpose. Is there a backend service > that consumes the metadata file and makes it queryable? > > It seems to me that we would want to have metadata query-able > across the different data stores (in much the same way that search > operates). This would indicate that we need to either have a single > central metadata service, and/or else federate the metadata across the > datastores and enforce some level of conformity across them. > > If we have an overarching metadata service or even a federated > metadata service, the other question that immediately comes up is > having single namespace that allows us to reference kbase objects > across the different datastores. For example, how do we distinguish > between something in the workspace that is called "kb|g.1716.peg.3960" > versus something in the cdmi called "kb|g.1716.peg.3960" versus a file > in Shock that may be named the same? > Do we use some kind of URI based approach, such as > shock://kbase.us/kb|g.1716.peg.3960 and > workspace://kbase.us/kb|g.1716.peg.3960 ? We can't really provide an > JSON-RPC url that is a accessor for the object the way we might with a > REST api. > > Anyway - just some more grist for the mill. > > Steve > > On Thu, Mar 14, 2013 at 8:53 AM, Folker Meyer <folker(a)anl.gov> wrote: >> Hi all, >> >> we have had several emails/discussions with several groups now on metadata in KBase. Now a couple of people are talking about using the API to store metadata. >> >> There is a lot of prior discussion of metadata and metadata standards outside KBase and I think it is important to find a solution that integrates well with the external standards community. And in addition, rather than having a solution creep into place, I think it would be good to get in front of this and define what it is that we (as KBase) want. >> >> As a result I mentioned in a recent discussion the fact that we have an implementation of this in MG-RAST that might be worth looking at. As a punishment for this I was asked to put together a document outlining our solution for this in MG-RAST (see attached). >> >> There are some questions that I see us faced with: >> >> - do we need a single metadata strategy for KBase >> - what aspects of metadata are required for each domain >> - what type of solution are people comfortable with >> - what are our customers comfortable with >> >> I am sure I am missing tons of stuff that we need to think about and agree on a solution for, but hopefully this will be food for thought. >> >> >> Best, >> Folker >> >> >> _______________________________________________ >> Kbase-devel mailing list >> Kbase-devel(a)lists.kbase.us >> https://lists.kbase.us/mailman/listinfo/kbase-devel >>

1 0

Re: [Kbase-MicroComm] Starting a discussion of metadata in KBase
by Marcin 14 Mar '13

14 Mar '13

Hi Folker, Thanks for sending this - Yes, there are lots of thoughts on metadata. As you may know MicrobesOnline also stores metadata, and I venture to say that what we have for our in-house experiments and what Gavin has implemented in the Experiment service is a helpful standard/ example. Some pieces of that standard are: - Standardization across data sets, meaning that the terminology and content are consistent for different experiments. I would say this goes beyond CV in that the metadata itself is structured - perhaps this can be thought of as ordered metadata with required fields. In effect this means the data are machine readable and can be incorporated in automation. BTW this also includes strain sequence information. - Related to the above, the same people performed the experiments and encoded the metadata, which is rather luxurious. - The experiment compendium actually 'tests' different aspects of the conditions and this is reflected in the metadata. - The metadata is complete enough that one can build a virtual media for modeling etc. Much of this overlaps with your document. I know, part of the above is experimental design for which we usually have no control, but it does present a sort of best case. Regarding the last point for modeling, it may be hard to create virtual environments but perhaps also worth thinking about, I mentioned in house experiments and there the situation is nice. For the rest of the data, e.g. imports from GEO, things are not as nice. In fact they can be pretty horrible, up to and including lines of free text with no controlled vocabulary. It would be great to solve this problem in general but that may be a significant effort. Really that means devising a system to determine condition terms (CV) from free text and ensuring the term extraction is meaningful. In addition this is somewhat domain specific with say microarray conditions being mostly orthogonal to environmental data etc. Here the question is how your proposal applies to existing data in KBase or data that is automatically imported in batch. These will likely form the bulk of KBase data in the immediate future. Based on experience it will be hard to get users to upload their data through an interface, but not impossible - this depends on the 'reward'. Clearly having consistent meta data is enabling for integration and analysis. For example, knowing the similarity/ difference in conditions between say a metabolic and a phenotype experiment is rich information. If anything we should at least have the ability to 'match' conditions, or even score metadata similarity - perhaps this could be done with free text as well but without the need for CV exactly. I do like the idea of different levels of metadata quality but am worried that entropy will drive most of the cases to a less useful state. I believe the main question is - what does KBase plan to do with the metadata for different data types? Is it mostly labels for the UIs and text search? Personally I imagine being able to link different datasets by their conditions (based on a score) and performing enrichment analysis on the metadata terms ... Marcin On Mar 14, 2013, at 8:53 AM, Folker Meyer <folker(a)anl.gov> wrote: > Hi all, > > we have had several emails/discussions with several groups now on metadata in KBase. Now a couple of people are talking about using the API to store metadata. > > There is a lot of prior discussion of metadata and metadata standards outside KBase and I think it is important to find a solution that integrates well with the external standards community. And in addition, rather than having a solution creep into place, I think it would be good to get in front of this and define what it is that we (as KBase) want. > > As a result I mentioned in a recent discussion the fact that we have an implementation of this in MG-RAST that might be worth looking at. As a punishment for this I was asked to put together a document outlining our solution for this in MG-RAST (see attached). > > There are some questions that I see us faced with: > > - do we need a single metadata strategy for KBase > - what aspects of metadata are required for each domain > - what type of solution are people comfortable with > - what are our customers comfortable with > > I am sure I am missing tons of stuff that we need to think about and agree on a solution for, but hopefully this will be food for thought. > > > Best, > Folker > > <TheMG-RASTMetadatasolution.pdf> > _______________________________________________ > Kbase-microcomm mailing list > Kbase-microcomm(a)lists.kbase.us > https://lists.kbase.us/mailman/listinfo/kbase-microcomm

1 0

no call today
by Dylan Chivian 14 Mar '13

14 Mar '13

thanks, - dylan

1 0

Re: [Kbase-MicroComm] [Kbase-devel] Starting a discussion of metadata in KBase
by Ware, Doreen 14 Mar '13

14 Mar '13

Thanks Folker I skimmed the document quickly and like this as a starting point. Plants will review this and add comments as it relates to the data and services we have touched. With regard to what our customers are comfortable with is a somewhat provocative question. What they may be comfortable with and what they need to be comfortable with may not be the same. Many users are not comfortable with using the CVs, and are seen as a necessary evil. Is there a central list of CV that are targeted to be supported now by KBase? Doreen From: kbase-devel-bounces(a)lists.kbase.us [mailto:[email protected]] On Behalf Of Folker Meyer Sent: Thursday, March 14, 2013 11:54 AM To: kbase-devel(a)lists.kbase.us developers Cc: kbase-microcomm(a)lists.kbase.us Subject: [Kbase-devel] Starting a discussion of metadata in KBase Hi all, we have had several emails/discussions with several groups now on metadata in KBase. Now a couple of people are talking about using the API to store metadata. There is a lot of prior discussion of metadata and metadata standards outside KBase and I think it is important to find a solution that integrates well with the external standards community. And in addition, rather than having a solution creep into place, I think it would be good to get in front of this and define what it is that we (as KBase) want. As a result I mentioned in a recent discussion the fact that we have an implementation of this in MG-RAST that might be worth looking at. As a punishment for this I was asked to put together a document outlining our solution for this in MG-RAST (see attached). There are some questions that I see us faced with: - do we need a single metadata strategy for KBase - what aspects of metadata are required for each domain - what type of solution are people comfortable with - what are our customers comfortable with I am sure I am missing tons of stuff that we need to think about and agree on a solution for, but hopefully this will be food for thought. Best, Folker _______________________________________________ Kbase-devel mailing list Kbase-devel(a)lists.kbase.us<mailto:[email protected]> https://lists.kbase.us/mailman/listinfo/kbase-devel

1 0

Starting a discussion of metadata in KBase
by Folker Meyer 14 Mar '13

14 Mar '13

Hi all, we have had several emails/discussions with several groups now on metadata in KBase. Now a couple of people are talking about using the API to store metadata. There is a lot of prior discussion of metadata and metadata standards outside KBase and I think it is important to find a solution that integrates well with the external standards community. And in addition, rather than having a solution creep into place, I think it would be good to get in front of this and define what it is that we (as KBase) want. As a result I mentioned in a recent discussion the fact that we have an implementation of this in MG-RAST that might be worth looking at. As a punishment for this I was asked to put together a document outlining our solution for this in MG-RAST (see attached). There are some questions that I see us faced with: - do we need a single metadata strategy for KBase - what aspects of metadata are required for each domain - what type of solution are people comfortable with - what are our customers comfortable with I am sure I am missing tons of stuff that we need to think about and agree on a solution for, but hopefully this will be food for thought. Best, Folker

1 0