Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A variety of improvements around tool parameter modeling (second try) #19027

Merged
merged 11 commits into from
Oct 29, 2024

Conversation

jmchilton
Copy link
Member

A redo of #18952 on top of the new tool execution testing stuff and with a few more fixes.

Old PR Description:

This contains tests, fixes, and refactorings of the tool parameter models aimed at enabling two applications/APIs - these applications are a tool landing API and the tool request API. Those APIs and corresponding runtimes are not included here but can be found in the structured tool state branch (tool request API and tool landing API).

The bulk of the enhancements around enabling tool landing are found in a single commit titled "models for tool landing request state...". This introduces two new tool state representation types and wide variety of parameter specification tests that describe how the API will work and how models will be stored. The API will consume tool state with the representation type "landing_request" and will store the representation "landing_request_internal" in the database. The internal version is the decoded version of that tool state. The idea behind the landing API is that essentially everything is optional - external entities could for instance host a collection of FASTA files and provide links to tools to align against these for instance - all the parameters but the database would be deferred to the users discretion. The parameters that are provided though will be validated.

The remainder of the commits are a collection of things that make more confident were not doing damage with the tool request API. Part of that effort is having a fully expanded representation of the tool state that describes something pretty close to what we feed the wrappers - defaults expanded out, no absent conditionals, etc... That effort resulting in me writing lots of API tests to describe how the current API works so we don't regress it, implementing validation of static parameter validators so we can mimic some of the runtime behavior of the tools and write stronger upfront validation of the requests, etc...

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

self.datalist = []
for title, value, _ in input_source.parse_static_options():
self.datalist.append({"label": title, "value": value})

# why does Integer and Float subclass this :_(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does it? :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know but the code sharing in this file is very problematic all over.

):
super().__init__(message, negate)
assert filename
assert os.path.exists(filename), f"File {filename} specified by the 'filename' attribute not found"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that really needed here as assert?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was present in the existing validation code. I guess the idea is to not load the tool if that file is not available.

lib/galaxy/tools/stock.py Outdated Show resolved Hide resolved
self._inputs = inputs
return self

# aliases for self to create silly little English sentense... inputs.when.flat().when.legacy()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:-)

@jmchilton jmchilton force-pushed the parameter_model_enhancements branch from 0a01326 to bf2f290 Compare October 19, 2024 14:34
Copy link
Member

@bgruening bgruening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jmchilton

@bgruening bgruening merged commit d11d893 into galaxyproject:dev Oct 29, 2024
53 of 54 checks passed
@galaxyproject-sentryintegration
Copy link

galaxyproject-sentryintegration bot commented Oct 29, 2024

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ KeyError: 'sift_db' galaxy.tool_util.data in __getitem__ View Issue
  • ‼️ KeyError: 'sift_db' galaxy.tool_util.data in __getitem__ View Issue
  • ‼️ ValueError: invalid literal for int() with base 10: 'inf' galaxy.tool_util.parser.parameter_validators in... View Issue
  • ‼️ ValueError: invalid literal for int() with base 10: 'inf' galaxy.tool_util.parser.parameter_validators in... View Issue

Did you find this useful? React with a 👍 or 👎

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Oct 31, 2024
- Allow specifying a min/max for integer and float parameters.
- Swap language from regex to "Regular Expression" and synchronize help with regular expression fields in rule builder.
- Extend regular expression help.
- Improve default message for in_range validator. Probably a reversion in some ways introduced with galaxyproject#19027.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Oct 31, 2024
- Allow specifying a min/max for integer and float parameters.
- Swap language from regex to "Regular Expression" and synchronize help with regular expression fields in rule builder.
- Extend regular expression help.
- Improve default message for in_range validator. Probably a reversion in some ways introduced with galaxyproject#19027.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 4, 2024
- Allow specifying a min/max for integer and float parameters.
- Swap language from regex to "Regular Expression" and synchronize help with regular expression fields in rule builder.
- Extend regular expression help.
- Improve default message for in_range validator. Probably a reversion in some ways introduced with galaxyproject#19027.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 7, 2024
- Allow specifying a min/max for integer and float parameters.
- Swap language from regex to "Regular Expression" and synchronize help with regular expression fields in rule builder.
- Extend regular expression help.
- Improve default message for in_range validator. Probably a reversion in some ways introduced with galaxyproject#19027.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 8, 2024
- Allow specifying a min/max for integer and float parameters.
- Swap language from regex to "Regular Expression" and synchronize help with regular expression fields in rule builder.
- Extend regular expression help.
- Improve default message for in_range validator. Probably a reversion in some ways introduced with galaxyproject#19027.
jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 13, 2024
- Allow specifying a min/max for integer and float parameters.
- Swap language from regex to "Regular Expression" and synchronize help with regular expression fields in rule builder.
- Extend regular expression help.
- Improve default message for in_range validator. Probably a reversion in some ways introduced with galaxyproject#19027.
@ahmedhamidawan ahmedhamidawan added the highlight/dev Included in admin/dev release notes label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants