Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a query for the tool-input-size-to-memory-usage ratio. #139

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

natefoo
Copy link
Member

@natefoo natefoo commented Nov 15, 2023

Hoping to use this for insight on TPV rules, although tools I've looked at so far tend not to have super meaningful correlations 🙃

@bgruening
Copy link
Member

@natefoo did you had a look at good-for-pulsar. Should we rename this one to fit into your naming schema and be a bit more explicit about what it does :) Should we harmonize here?

@natefoo
Copy link
Member Author

natefoo commented Nov 15, 2023

Ah thanks for the heads up, looks useful - although that is input-size-to-runtime, but they could certainly be combined.

@bgruening
Copy link
Member

or we rename good-for-pulsar to tool-input-size-to-runtime or something like that.

@hexylena
Copy link
Member

yes it's a good suggestion, please rename the pulsar function to match.

CHANGELOG.md Outdated Show resolved Hide resolved
limit_clause="ORDER BY j.id DESC LIMIT $arg_last"
fi
if [[ -n "$arg_summary" ]]; then
summary="$(summary_statistics 'jmn.metric_value / isz.bytes' 0)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so glad stuff like this can get re-used

@natefoo
Copy link
Member Author

natefoo commented Nov 22, 2023

The whole time I was writing this I was like "I swear I already wrote this," but I looked through the existing queries and didn't see it.

Well... #78 🤦

@natefoo natefoo marked this pull request as draft September 12, 2024 17:54
@natefoo
Copy link
Member Author

natefoo commented Sep 12, 2024

This is better than the other query IMO but needs a bit more work and to integrate the two of them.

@natefoo
Copy link
Member Author

natefoo commented Nov 7, 2024

Another thing to note - it would be great if we could determine (or barring that, estimate) the uncompressed input size, since that is probably wildly varying the data.

Related discussion in galaxyproject/total-perspective-vortex#141, but unlike TPV we are collecting historical data about inputs that are probably already deleted, so we can't check the actual size (and that would be impractically slow anyway).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants