Yes using the pre computed metagenome analysis as an index into the reads is the right idea IMO

Sent from mobile device

On May 16, 2013, at 19:08, "Rick @ Argonne" <stevens@anl.gov> wrote:

One approach (im sure there ate variations and better) to the clever case could be resolved by finding a set of more loosely related genomes gathering the superset then a sequence search in that smaller set. 

Ideally you would be searching contigs or representatives rather than raw MG reads. So it would be tractable. But I bet we would almost never be in this case. 

Sent from my iPhone

On May 16, 2013, at 18:52, "Rick @ Argonne" <stevens@anl.gov> wrote:

I would build tables that map metagenome IDs to the IDs for isolate genomes in the CDM and vice versa. 

If one derefrenced via the M5nr, the M5rna and silva 

Then the vast majority would be captured by the normal annotation process of each domain. 

That leaves the case where something is too new to have been captured in the M5s

If the new genome is a close relative then a simple transitive search works. 

If its really different then one would need to do something more clever. 

I would focus on the non clever cases first which will cover 99.99% of the cases. 



Sent from my iPhone

On May 16, 2013, at 18:37, Dylan Chivian <dcchivian@lbl.gov> wrote:

what is all this about a use-case?  it's the same thing we've been asking for from you since very early in the project.  we want to be able to say which metagenomes a given KBase genome can be found in (or a close relative).  it's pretty straightforward.  if I just state it like that, how would you implement a solution?

thanks,
- dylan



On Thu, May 16, 2013 at 3:39 PM, Folker Meyer <folker@anl.gov> wrote:
ah. I see so this for a use case like RAST?

as Silva's job is to mine all databases for 16s and expose them to the rest of us, I assume this would be a novel thing. So yes if it is not in the database we will not be able to search against it.

In any case my comment is even more applicable: I don't understand your use-case well enough to be helpful.

best,
Folker

On May 16, 2013, at 5:34 PM, Dylan Chivian <dcchivian@lbl.gov> wrote:

> "user-defined" in this case means a 16S from a KBase microbial genome (that isn't necessarily in GenBank or SILVA or the m5rnaNR due to the inability to synchronize it with KBase regularly as we've discussed).
>
> thanks,
> - dylan
>
>
> On Thu, May 16, 2013 at 3:28 PM, Folker Meyer <folker@anl.gov> wrote:
>
> On May 16, 2013, at 5:24 PM, Dylan Chivian <dcchivian@lbl.gov> wrote:
>
> > Sorry, which matching?  (a) metagenome gene to 16S database or (b) metagenome read to user-defined 16S query sequence?
>
> Now you are loosing me. Why would the user want to define a 16s query sequence? It seems clear to me that I do not understand your use-case.
>
> The API is providing matches that were computed against the m5nr and m5rnaNR databases.
>
> If you want to search against another database that is not part of the API. However it is very likely that the other 16s sequence will be in our database. once we are fully in shock you will be able to ask for the top 300 hits for a read/cluster if that is what you want. this may or may not be interesting for your use-case.
>
> >
> > - dylan
> >
> >
> >
> > On Thu, May 16, 2013 at 3:21 PM, Rick stevens <stevens@anl.gov> wrote:
> > The matching has already been computed.
> >
> > Sent from my iPad
> >
> > On May 16, 2013, at 5:15 PM, Dylan Chivian <dcchivian@lbl.gov> wrote:
> >
> >> ... and there's no sequence-based search query, right?  we'll need to do the alignments ourselves?
> >>
> >> Thanks
> >> - Dylan
> >>
> >>
> >> On Thu, May 16, 2013 at 3:12 PM, Dylan Chivian <dcchivian@lbl.gov> wrote:
> >> Great!  What's the command for getting the public metagenome IDs?
> >>
> >> Thanks,
> >> - Dylan
> >>
> >>
> >> On Thu, May 16, 2013 at 2:45 PM, Folker Meyer <folker.meyer@gmail.com> wrote:
> >> sure. this is quite simple in the API.
> >>
> >>
> >> pseudo code:
> >> -----------
> >> for each public_metagenome
> >>
> >> do
> >>  # get abundance profile
> >>   curl "http://api.metagenomics.anl.gov/abundanceprofile/ID?type=organism&source=Genbank" | json_xs
> >>
> >> # e.g. curl "http://api.metagenomics.anl.gov/abundanceprofile/mgm4440026.3?type=organism&source=Genbank" | json_xs
> >>
> >> done
> >> -----------
> >> This gives you the taxon info derived from either proteins or 16s using Genbank annotations and NCBI taxon IDs+Names. SEED annotations are also mapped to NCBI taxonomy.
> >>
> >> Using Silva (aka SSU) as namespace instead of GenBank above will just give you 16s.
> >>
> >> best,
> >> Folker
> >>
> >>
> >> On May 16, 2013, at 4:19 PM, Matthew Henderson <mhenderson@lbl.gov> wrote:
> >>
> >> > Hi Folker,
> >> >
> >> > Right now we are just building prototypes, so it would still be useful to have the functionality even if it is not optimized.
> >> >
> >> > Can we see an example of how to do this with the api?  I think that is all we are asking for at this point.
> >> >
> >> > Thanks,
> >> > Matt
> >> >
> >> >
> >> > On Thu, May 16, 2013 at 3:34 PM, Folker Meyer <folker.meyer@gmail.com> wrote:
> >> > this is something that is supported by the API now. But not fast.
> >> >
> >> > The query is one of the uses cases we are re-designing the search for MG-RAST for. We expect it to be done in a couple (read 2) of weeks.
> >> >
> >> > once that is done, we will implement simple searches in MG-RAST like which taxon is present in which geographic region etc.
> >> >
> >> > currently it is very slow. you are going 90 degrees to the way the system is optimized
> >> >
> >> > Best,
> >> > Folker
> >> >
> >> > On May 16, 2013, at 3:17 PM, Paramvir Dehal <psdehal@lbl.gov> wrote:
> >> >
> >> > > Yeah, those methods would work, I just need to know the function call and how to execute it.
> >> > >
> >> > >
> >> > > On Thu, May 16, 2013 at 3:14 PM, Rick stevens <stevens@anl.gov> wrote:
> >> > > For isolate genomes we can use metadata and 16s precomputed mapping.
> >> > >
> >> > > For WGS  and WGS contigs we need to workout precisely what you mean by assign to a taxon grouping.
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Sent from my iPad
> >> > >
> >> > > On May 16, 2013, at 2:58 PM, Rick stevens <stevens@anl.gov> wrote:
> >> > >
> >> > >> Dylan
> >> > >>
> >> > >> I'm trying Ti understand what you want to build.
> >> > >>
> >> > >> I think you want some way to have some one select a taxon and bring up in a taxon page all genomes and metagenomes and metagenome derived contigs associated with that taxon..
> >> > >>
> >> > >> Is that correct ?
> >> > >>
> >> > >> Sent from my iPad
> >> > >>
> >> > >> On May 16, 2013, at 2:26 PM, Dylan Chivian <dcchivian@lbl.gov> wrote:
> >> > >>
> >> > >>> I'm confused.  Isn't this week a joint hack-a-thon?  My guys flew out there to work with your guys.  Is that not happening?
> >> > >>>
> >> > >>> Thanks,
> >> > >>> - Dylan
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> On Thu, May 16, 2013 at 12:21 PM, Folker Meyer <folker@mcs.anl.gov> wrote:
> >> > >>> We are super busy right now. Can we do this next week over the phone. I still need some input on the actual goals of this.
> >> > >>>
> >> > >>>
> >> > >>> --
> >> > >>> Sent from mobile device
> >> > >>>
> >> > >>> On May 16, 2013, at 14:14, Matthew Henderson <mhenderson@lbl.gov> wrote:
> >> > >>>
> >> > >>>> Paramvir, Mike and I are at Freund Lodge currently, and you guys (Andreas?) are welcome to come over and be part of the excitement of the KBase build and "stuff".  Mike and I will also be here tomorrow morning, but will have to leave around lunchtime to get back to the airport for the trip back home.
> >> > >>>>
> >> > >>>> Paramvir is less concerned at the moment about thresholding and just wants to see some functionality.
> >> > >>>>
> >> > >>>> Alternatively, if you are not feeling Freundlich, an email pointing us in the proper direction would also be appreciated.
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> Matt
> >> > >>>>
> >> > >>>>
> >> > >>>> On Thu, May 16, 2013 at 1:28 PM, Dylan Chivian <dcchivian@lbl.gov> wrote:
> >> > >>>> Paramvir is a stakeholder here, so we should get his input, including on which identity thresholds he'd like to see.
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> - Dylan
> >> > >>>>
> >> > >>>>
> >> > >>>> On Thu, May 16, 2013 at 11:10 AM, Dylan Chivian <dcchivian@lbl.gov> wrote:
> >> > >>>> Hi guys,
> >> > >>>>
> >> > >>>> instead of the call today, probably the time would be best spent with Folker and Andreas working with Matt and Mike directly on the metagenomics species search functionality for the Taxon pages.  I expect it is an expensive operation, so it is probably something that should be a one-time compute and stored somewhere associated with each taxon rather than on-the-fly when someone calls up the Taxon page.  There should probably also be separate lists for 99% identity and 97% identity (not sure other thresholds make any sense).
> >> > >>>>
> >> > >>>> Thoughts?  Are there other things to discuss and we should have the call anyway?
> >> > >>>>
> >> > >>>> Thanks,
> >> > >>>> - Dylan
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> _______________________________________________
> >> > >>>> Kbase-microcomm mailing list
> >> > >>>> Kbase-microcomm@lists.kbase.us
> >> > >>>> https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> > >>>>
> >> > >>>>
> >> > >>>> _______________________________________________
> >> > >>>> Kbase-microcomm mailing list
> >> > >>>> Kbase-microcomm@lists.kbase.us
> >> > >>>> https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> > >>>
> >> > >>> _______________________________________________
> >> > >>> Kbase-microcomm mailing list
> >> > >>> Kbase-microcomm@lists.kbase.us
> >> > >>> https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> > >>>
> >> > >>>
> >> > >>> _______________________________________________
> >> > >>> Kbase-microcomm mailing list
> >> > >>> Kbase-microcomm@lists.kbase.us
> >> > >>> https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> > >
> >> > > _______________________________________________
> >> > > Kbase-microcomm mailing list
> >> > > Kbase-microcomm@lists.kbase.us
> >> > > https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> > >
> >> > >
> >> > > _______________________________________________
> >> > > Kbase-microcomm mailing list
> >> > > Kbase-microcomm@lists.kbase.us
> >> > > https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> >
> >> > _______________________________________________
> >> > Kbase-microcomm mailing list
> >> > Kbase-microcomm@lists.kbase.us
> >> > https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >> >
> >> > _______________________________________________
> >> > Kbase-microcomm mailing list
> >> > Kbase-microcomm@lists.kbase.us
> >> > https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >>
> >> _______________________________________________
> >> Kbase-microcomm mailing list
> >> Kbase-microcomm@lists.kbase.us
> >> https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >>
> >>
> >> _______________________________________________
> >> Kbase-microcomm mailing list
> >> Kbase-microcomm@lists.kbase.us
> >> https://lists.kbase.us/mailman/listinfo/kbase-microcomm
> >
> > _______________________________________________
> > Kbase-microcomm mailing list
> > Kbase-microcomm@lists.kbase.us
> > https://lists.kbase.us/mailman/listinfo/kbase-microcomm
>
>
> _______________________________________________
> Kbase-microcomm mailing list
> Kbase-microcomm@lists.kbase.us
> https://lists.kbase.us/mailman/listinfo/kbase-microcomm


_______________________________________________
Kbase-microcomm mailing list
Kbase-microcomm@lists.kbase.us
https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________
Kbase-microcomm mailing list
Kbase-microcomm@lists.kbase.us
https://lists.kbase.us/mailman/listinfo/kbase-microcomm
_______________________________________________
Kbase-microcomm mailing list
Kbase-microcomm@lists.kbase.us
https://lists.kbase.us/mailman/listinfo/kbase-microcomm