Job output transform #548
Job output transform #548
Conversation
Direct Links in job output + makefile + tests
Testing transformer engine added
Remove requirements-trfm Add to main requirements
Why not pandas
Update requirements
Ajout Multi frame Images
github-actions
bot
added
ci/doc
There was a problem hiding this comment.
There's a few linting issues that can be resolved automatically using the make fix[...] targets.
Validate as well using make check[...] targets any other flagged issues that might need some manual adjustments. I didn't comment on all of them.
I did not go in depth into the actual implementation of the transform functions.
I think there is a few tests missing to validate that the actual results from each transformation is valid (eg: pixel values from one format to another are properly rendered or expected text from one format to another is properly structured).
github-actions
bot
added
feature/job
github-actions
bot
added
the
feature/cli
github-actions
bot
added
the
feature/db
github-actions
bot
added
the
feature/providers
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #548 +/- ##
==========================================
- Coverage 87.07% 87.04% -0.04%
==========================================
Files 80 84 +4
Lines 20472 21134 +662
Branches 2784 2898 +114
==========================================
+ Hits 17827 18397 +570
- Misses 1902 1970 +68
- Partials 743 767 +24 ☔ View full report in Codecov by Sentry. |
| @@ -12,6 +12,13 @@ Changes | |||
|
|
|||
| Changes: | |||
| -------- | |||
| - Add support for various GeoTIFF formats, allowing flexible handling and representation of GeoTIFFs in outputs | |||
| (fixes `#100 <https://github.com/crim-ca/weaver/issues/100>`_). | |||
| - Add support for ``GET /results/{id}`` and `` GET /outputs/{id}`` routes to enable direct access to individual | |||
There was a problem hiding this comment.
Update paths to start with the /jobs/{jobId}/...
Remove the extra space between `` and GET causing bad parsing.
| - Add support for various GeoTIFF formats, allowing flexible handling and representation of GeoTIFFs in outputs | ||
| (fixes `#100 <https://github.com/crim-ca/weaver/issues/100>`_). | ||
| - Add support for ``GET /results/{id}`` and `` GET /outputs/{id}`` routes to enable direct access to individual | ||
| job result items by ID. This enhancement includes: support alternate representations based on the Accept header. |
There was a problem hiding this comment.
- remove
: - add `` to
Accept
| Link headers containing all possible output formats, allowing retrieval via query parameters | ||
| (e.g., output?f=application/x-yaml). (fixes `#18 <https://github.com/crim-ca/weaver/issues/18>`_). |
There was a problem hiding this comment.
Make this a separate item, and apply Link formatting, e.g.:
"- Return Link headers all possible alternate output formats, ..."
Apply the formatting to the example, and use the full /jobs/{jobId}/... link.
| # Get requested media-type. "*/*" if omit | ||
| accept = str(request.accept) if request.accept else "*/*" | ||
| headers = request.headers | ||
| possible_media_types = get_job_possible_output_formats(job) | ||
| possible_media_types = next((o["alternatives"] for o in possible_media_types | ||
| if str(o["output_id"]) == output_id), None) |
There was a problem hiding this comment.
Do the media-type lookup after validating that the result ID is actually available (code just below).
Otherwise, these are unnecessary operations.
Also, this seems over-complicated with the function.
It creates an alternate dict of metadata, just to not use all the keys and return only the media-types.
It would be easier to simply do the loop here and set the variable with the matched combination of output-id and accept media-type that apply for the specific output transforms.
| "code": "", | ||
| "description": "The requested output format is not in the possible output formats.", | ||
| "cause": "Incompatible mime Types", | ||
| "error": "", | ||
| "value": "" |
There was a problem hiding this comment.
Improve the details. This is misleading. The error is related to output ID not available, not the format.
| "value": "" | ||
| } | ||
| ) | ||
| result_media_type = result["mimeType"] |
There was a problem hiding this comment.
use get_field to handle case where mediaType name would used instead
| result_media_type = guess_target_format(request, default=result_media_type) | ||
|
|
||
| # if format requested not equal to result media type and not in possible mediatypes... | ||
| if accept != result_media_type and accept not in possible_media_types: |
There was a problem hiding this comment.
no need to cross-validate accept, this is what guess_target_format does (+ format queries)
directly do if result_media_type not in possible_media_types
(fixes
#100 <https://github.com/crim-ca/weaver/issues/100>_).GET /results/{id}andGET /outputs/{id}routes to enable direct access to individualjob result items by ID. This enhancement includes: support alternate representations based on the Accept header.
If an alternate format (e.g., YAML for a JSON source) is requested it will be automatically generated and returned.
Link headers containing all possible output formats, allowing retrieval via query parameters
(e.g., output?f=application/x-yaml). (fixes
#18 <https://github.com/crim-ca/weaver/issues/18>_).