limit support?

[bicycle1885]

Hi, I'm interested in using this interface to access biological annotation databases. Databases may be located in a local machine or over the Internet. In such a case, we often want to retrieve a small part of results to quickly check whether the query works as expected. If it takes a long time to retrieve all records, I guess we want something like head (or tail) verb to limit the number of maximum records.

What I have in mind is something like this:

@query humanDB |> filter(chromosome == "12") |> select(gene, locus) |> head(10)

Do you think it's worth adding to the verbs?

[nalimilan]

Could be useful. Though there have been lots of conflicts between different packages defining this function. We should either add it to Julia Base or find an appropriate home for it (maybe AbstractTables?).

[johnmyleswhite]

Maybe this should be something like limit? Some large-scale databases can't support an option like head or tail since the result set isn't ordered. They're only able to short-circuit accumulation of the result set.

[bicycle1885]

I think there is no need to define names like head or tail in this package directly because queries are parsed in the @query macro. What we really need would be just a node that limits the number of records. limit may be a better name since it has a good correspondence to SQL's LIMIT clause.

[davidagold]

I agree something like this should be supported. There could be a limit verb. Another option, which I've played with elsewhere, is to include limit and offset as kwargs to collect.

[davidagold]

I agree something like this should be supported. There could be a limit
verb. Another option, which I've played with elsewhere, is to include limit
and offset as kwargs to collect.

On Tuesday, October 25, 2016, John Myles White [email protected]
wrote:

Maybe this should be something like limit? Some large-scale databases
can't support an option like head or tail since the result set isn't
ordered. They're only able to short-circuit accumulation of the result set.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#28 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALUCm5KiEfLnmogl4I6H_CJpdKrt4ZgXks5q3nvkgaJpZM4KggPp
.

[bicycle1885]

But we may want to limit subqueries when joining some queries. Adding the limit argument to collect can't do that, can it?

[davidagold]

Sorry for my delay in addressing your comment. I cite graduate school. You make a very good point. I'm now in favor of including LimitNode and OffsetNode.