As Rick points out, the problem is one then needs to check whether the transitive similarity is still within threshold (i.e. KB_16S - m5_16S >= 97% and m5_16S - MG_16S >= 97% does not necessarily mean KB_16S - MG_16S >= 97%) and also you could miss stuff in the other direction where the KB_16S is closer to the MG_16S. I think you just draw a 2x radius circle around the KB_16S and then realign the KB_16S against the MG_16S and check identity. So if you want to be within 97%, then the KB_16S - m5_16S >= 94% (i.e. within 6%), collect the set of MG_16S that satisfy m5_16S - MG_16S >= 97% (that should be a Communities_API service), and then check KB_16S - MG_16S >= 97% (i.e. within 3%). Is my math right? Is there a query to return all sequences (or maybe just the abundance counts) within thresholding of identity, e-value, length to a given m5 reference sequence? Thanks, - Dylan On Thu, May 16, 2013 at 5:26 PM, Folker Meyer <[email protected]> wrote:
Yes using the pre computed metagenome analysis as an index into the reads is the right idea IMO
Sent from mobile device
On May 16, 2013, at 19:08, "Rick @ Argonne" <[email protected]> wrote:
One approach (im sure there ate variations and better) to the clever case could be resolved by finding a set of more loosely related genomes gathering the superset then a sequence search in that smaller set.
Ideally you would be searching contigs or representatives rather than raw MG reads. So it would be tractable. But I bet we would almost never be in this case.
Sent from my iPhone
On May 16, 2013, at 18:52, "Rick @ Argonne" <[email protected]> wrote:
I would build tables that map metagenome IDs to the IDs for isolate genomes in the CDM and vice versa.
If one derefrenced via the M5nr, the M5rna and silva
Then the vast majority would be captured by the normal annotation process of each domain.
That leaves the case where something is too new to have been captured in the M5s
If the new genome is a close relative then a simple transitive search works.
If its really different then one would need to do something more clever.
I would focus on the non clever cases first which will cover 99.99% of the cases.
Sent from my iPhone
On May 16, 2013, at 18:37, Dylan Chivian <[email protected]> wrote:
what is all this about a use-case? it's the same thing we've been asking for from you since very early in the project. we want to be able to say which metagenomes a given KBase genome can be found in (or a close relative). it's pretty straightforward. if I just state it like that, how would you implement a solution?
thanks, - dylan
On Thu, May 16, 2013 at 3:39 PM, Folker Meyer <[email protected]> wrote:
ah. I see so this for a use case like RAST?
as Silva's job is to mine all databases for 16s and expose them to the rest of us, I assume this would be a novel thing. So yes if it is not in the database we will not be able to search against it.
In any case my comment is even more applicable: I don't understand your use-case well enough to be helpful.
best, Folker
On May 16, 2013, at 5:34 PM, Dylan Chivian <[email protected]> wrote:
"user-defined" in this case means a 16S from a KBase microbial genome (that isn't necessarily in GenBank or SILVA or the m5rnaNR due to the inability to synchronize it with KBase regularly as we've discussed).
thanks, - dylan
On Thu, May 16, 2013 at 3:28 PM, Folker Meyer <[email protected]> wrote:
On May 16, 2013, at 5:24 PM, Dylan Chivian <[email protected]> wrote:
Sorry, which matching? (a) metagenome gene to 16S database or (b) metagenome read to user-defined 16S query sequence?
Now you are loosing me. Why would the user want to define a 16s query sequence? It seems clear to me that I do not understand your use-case.
The API is providing matches that were computed against the m5nr and m5rnaNR databases.
If you want to search against another database that is not part of the API. However it is very likely that the other 16s sequence will be in our database. once we are fully in shock you will be able to ask for the top 300 hits for a read/cluster if that is what you want. this may or may not be interesting for your use-case.
- dylan
On Thu, May 16, 2013 at 3:21 PM, Rick stevens <[email protected]>
wrote:
The matching has already been computed.
Sent from my iPad
On May 16, 2013, at 5:15 PM, Dylan Chivian <[email protected]> wrote:
... and there's no sequence-based search query, right? we'll need to do the alignments ourselves?
Thanks - Dylan
On Thu, May 16, 2013 at 3:12 PM, Dylan Chivian <[email protected]> wrote: Great! What's the command for getting the public metagenome IDs?
Thanks, - Dylan
On Thu, May 16, 2013 at 2:45 PM, Folker Meyer < [email protected]> wrote: sure. this is quite simple in the API.
pseudo code: ----------- for each public_metagenome
do # get abundance profile curl " http://api.metagenomics.anl.gov/abundanceprofile/ID?type=organism&source=Gen..." | json_xs
# e.g. curl " http://api.metagenomics.anl.gov/abundanceprofile/mgm4440026.3?type=organism&..." | json_xs
done ----------- This gives you the taxon info derived from either proteins or 16s using Genbank annotations and NCBI taxon IDs+Names. SEED annotations are also mapped to NCBI taxonomy.
Using Silva (aka SSU) as namespace instead of GenBank above will just give you 16s.
best, Folker
On May 16, 2013, at 4:19 PM, Matthew Henderson <[email protected]> wrote:
Hi Folker,
Right now we are just building prototypes, so it would still be useful to have the functionality even if it is not optimized.
Can we see an example of how to do this with the api? I think that is all we are asking for at this point.
Thanks, Matt
On Thu, May 16, 2013 at 3:34 PM, Folker Meyer < [email protected]> wrote: this is something that is supported by the API now. But not fast.
The query is one of the uses cases we are re-designing the search for MG-RAST for. We expect it to be done in a couple (read 2) of weeks.
once that is done, we will implement simple searches in MG-RAST like which taxon is present in which geographic region etc.
currently it is very slow. you are going 90 degrees to the way the system is optimized
Best, Folker
On May 16, 2013, at 3:17 PM, Paramvir Dehal <[email protected]> wrote:
Yeah, those methods would work, I just need to know the function call and how to execute it.
On Thu, May 16, 2013 at 3:14 PM, Rick stevens <[email protected]> wrote: For isolate genomes we can use metadata and 16s precomputed mapping.
For WGS and WGS contigs we need to workout precisely what you mean by assign to a taxon grouping.
Sent from my iPad
On May 16, 2013, at 2:58 PM, Rick stevens <[email protected]> wrote:
Dylan
I'm trying Ti understand what you want to build.
I think you want some way to have some one select a taxon and bring up in a taxon page all genomes and metagenomes and metagenome derived contigs associated with that taxon..
Is that correct ?
Sent from my iPad
On May 16, 2013, at 2:26 PM, Dylan Chivian <[email protected]> wrote:
I'm confused. Isn't this week a joint hack-a-thon? My guys flew out there to work with your guys. Is that not happening?
Thanks, - Dylan
On Thu, May 16, 2013 at 12:21 PM, Folker Meyer < [email protected]> wrote: We are super busy right now. Can we do this next week over the phone. I still need some input on the actual goals of this.
-- Sent from mobile device
On May 16, 2013, at 14:14, Matthew Henderson < [email protected]> wrote:
Paramvir, Mike and I are at Freund Lodge currently, and you guys (Andreas?) are welcome to come over and be part of the excitement of the KBase build and "stuff". Mike and I will also be here tomorrow morning, but will have to leave around lunchtime to get back to the airport for the trip back home.
Paramvir is less concerned at the moment about thresholding and just wants to see some functionality.
Alternatively, if you are not feeling Freundlich, an email pointing us in the proper direction would also be appreciated.
Thanks, Matt
On Thu, May 16, 2013 at 1:28 PM, Dylan Chivian < [email protected]> wrote: Paramvir is a stakeholder here, so we should get his input, including on which identity thresholds he'd like to see.
Thanks, - Dylan
On Thu, May 16, 2013 at 11:10 AM, Dylan Chivian < [email protected]> wrote: Hi guys,
instead of the call today, probably the time would be best spent with Folker and Andreas working with Matt and Mike directly on the metagenomics species search functionality for the Taxon pages. I expect it is an expensive operation, so it is probably something that should be a one-time compute and stored somewhere associated with each taxon rather than on-the-fly when someone calls up the Taxon page. There should probably also be separate lists for 99% identity and 97% identity (not sure other thresholds make any sense).
Thoughts? Are there other things to discuss and we should have the call anyway?
Thanks, - Dylan
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________ Kbase-microcomm mailing list [email protected] https://lists.kbase.us/mailman/listinfo/kbase-microcomm