feat: implement the file functions from the WDL standard library. #254
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit implements the `basename` function from the WDL standard library.
This commit implements the `join_paths` function from the WDL standard library. It also exposes some of the well-known types from `STDLIB` in `wdl-analysis` for use in `wdl-engine`.
This commit implements the `glob` function from the WDL standard library. It also refactors out the arguments to the function callbacks into a `CallContext` struct so that the callbacks always receive one argument; the argument coercion has also been simplified in each function's implementation.
This commit implements the `size` function in the WDL standard library. Also fixes a bug in the signatures of `size` to allow passing a literal `None` without an ambiguity. Also fixes an incorrect panic in `as_string` implementations.
This was a regression when adding support for URLs to the CLI.
This commit implements the `stdout` and `stderr` functions from the WDL standard library.
This commit implements the `read_string` function from the WDL standard library.
This commit implements the `read_int` function from the WDL standard library.
This commit implements the `read_float` function from the WDL standard library.
This commit implements the `read_boolean` function from the WDL standard library.
This commit implements the `read_lines` function from the WDL standard library.
This commit implements the `write_lines` function from the WDL standard library. Additionally, this refactors the evaluation tests to make it easier to create temporary files and use a specified temporary directory for functions that create files.
This commit implements the `read_tsv` function from the WDL standard library.
This commit refactors the standard library implementation such that each function's implementation gets its own module.
This commit implements the `write_tsv` function from the WDL standard library. Additionally, it fixes some overloads of `read_tsv` and `write_tsv` that should be marked as minimum version 1.2. The constraints on the file functions that take arrays of structs should also be further constrained on the requirement that the structs only contain primitive members.
This commit implements the `read_map` function from the WDL standard library.
This commit implements the `write_map` function from the WDL standard library.
This commit implements the `read_json` function from the WDL standard library. It also moves `is_ident` into `wdl-grammar`.
This commit implements the `write_json` function from the WDL standard library. Additionally, it fixes two issues in `wdl-analysis`: the JSON-serializable constraint should allow any map key type that is coercible to string and object literal expressions should type check the member expressions of the literal. This also implements serializing values to JSON.
This commit replaces the implementation of `Value::from_json` with a new `deserialize` method that will work for any self-describing serde deserializer. It also makes the API consistent with serialization.
This commit implements the `read_object` function from the WDL standard library.
This commit implements the `read_objects` function from the WDL standard library. It also changes the storage of array values to be simply `Arc<Vec<Value>>`.
This commit implements the `write_object` function from the WDL standard library. It also corrects other TSV writing functions to ensure headers and values do not themselves contain tab characters.
This commit implements the `write_objects` function from the WDL standard library. It also refactors writing TSV values into a helper function.
|
The tests have some Windows-path-specific logic to them that I haven't tested on Windows yet, so the Windows CI might fail; I'll investigate if it does. |
|
So gauntlet is actually missing a diagnostic that should have been previously emitted. The problematic line is: disk_size = size(flatten(hifi_reads), "GB") + 10Where This was actually fixed in the upstream repository by wrapping the Looking into which specific change in this PR caused gauntlet to fail (in a good way). |
|
The windows failures are due to path separator issues; I'll get that cleaned up. |
|
After determining that none of these changes should have affected the gauntlet diagnostic, I can confirm that I'm seeing the missing diagnostic in |
Additionally, this changes gauntlet's output to not duplicate the duration of analysis on each file, since it's the same for every file in the repo (all files are analyzed as one). The duration is now printed out with the footer.
claymcleod
reviewed
Nov 19, 2024
claymcleod
approved these changes
Nov 19, 2024
a-frantz
reviewed
Nov 19, 2024
Co-authored-by: Andrew Frantz <[email protected]>
a-frantz
reviewed
Nov 19, 2024
Remove unnecessary `trim_end_matches` calls.
a-frantz
reviewed
Nov 19, 2024
peterhuene
changed the title
Fix comments.
a-frantz
reviewed
Nov 19, 2024
a-frantz
approved these changes
Nov 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements the following functions from the WDL standard library:
basenamejoin_pathsglobsizestdoutstderrread_stringread_intread_floatread_booleanread_lineswrite_linesread_tsvwrite_tsvread_mapwrite_mapread_jsonwrite_jsonread_objectread_objectswrite_objectwrite_objectsIt also corrects some issues in the definitions of the functions in
wdl-analysisand constraining certain generic function parameter types to structs with only primitive members.The function call implementation in
wdl-enginehas been refactored so that the current working directory and temp directory and available to the call context; consequently, the function tests inwdl-enginehave been refactored to make it easier to create files in a temporary directory.The function implementations themselves have been refactored into their own modules so as to make the implementations/tests isolated from one another.
Before submitting this PR, please make sure:
changes (when appropriate).
CHANGELOG.md(see"keep a changelog" for more information).