Metadata quality verification API #1874

sir-sigurd · 2020-10-30T18:47:02Z

Description

This adds workflow parameter to Package.push() and Package.build() functions.

TODO

Decide on workflow values for 'don't use workflow' and 'use default'
Store the used workflow ID and URL with versionID
Unit tests
Documentation
- Update docstrings
- Document workflows/config.yml
- Run build.py for new docstrings
Changelog entry
Move JSON Schema for config validation to its own file

codecov-io · 2020-10-31T13:17:09Z

Codecov Report

Merging #1874 (a02ee21) into master (1da0973) will increase coverage by 0.49%.
The diff coverage is 98.38%.

@@            Coverage Diff             @@
##           master    #1874      +/-   ##
==========================================
+ Coverage   89.86%   90.35%   +0.49%     
==========================================
  Files          62       64       +2     
  Lines        7411     7757     +346     
==========================================
+ Hits         6660     7009     +349     
+ Misses        751      748       -3

Flag	Coverage Δ
api-python	`89.28% <98.38%> (+0.84%)`	⬆️
lambda	`92.40% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
api/python/setup.py	`0.00% <ø> (ø)`
api/python/quilt3/util.py	`82.68% <44.44%> (+0.83%)`	⬆️
api/python/quilt3/workflows/__init__.py	`98.91% <98.91%> (ø)`
api/python/quilt3/backends/base.py	`89.56% <100.00%> (+0.23%)`	⬆️
api/python/quilt3/data_transfer.py	`81.39% <100.00%> (+0.35%)`	⬆️
api/python/quilt3/packages.py	`92.35% <100.00%> (+0.26%)`	⬆️
api/python/tests/integration/test_packages.py	`100.00% <100.00%> (ø)`
api/python/tests/integration/test_workflows.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1da0973...a02ee21. Read the comment docs.

api/python/quilt3/workflows/config-1.schema.json

api/python/quilt3/packages.py

docs/SUMMARY.md

docs/API Reference/Package.md

docs/Advanced Features/workflows.md

api/python/quilt3/workflows/config-1.schema.json

akarve · 2020-11-10T21:48:32Z

api/python/quilt3/workflows/config-1.schema.json

+  "additionalProperties": false,
+  "properties": {
+    "version": {
+      "const": "1"


Do we want semver for this?

I don't quite understand how it works for schemas 🤷‍♂️. I think I need to look for example.

Well at a bare minimum, make it a string?

It's already a string.

api/python/quilt3/workflows/config-1.schema.json

api/python/tests/integration/test_workflows.py

docs/Advanced Features/workflows.md

akarve · 2020-11-10T22:39:17Z

docs/Advanced Features/workflows.md

+*New in Quilt 3.3*
+
+`quilt3` provides API to ensure quality of package metadata. To start use it you need to create
+configuration file in your registry under `.quilt/workflows/config.yml`. For example:


Shouldn't we provide this as a constant from quilt3, or from an API?

I think that currently we don't have a good place for it.

What about util.py? See https://docs.quiltdata.com/api-reference/cli#constants-see-util-py-for-more

docs/Advanced Features/workflows.md

akarve · 2020-11-10T22:41:39Z

I will do more testing once we get this on stage, as well as some wordsmithing on the docs, but overall looks good. Added some comments and suggestions.

Co-authored-by: Aneesh Karve <[email protected]>

sir-sigurd force-pushed the workflows branch 5 times, most recently from 4ef658c to 6c42504 Compare November 1, 2020 15:42

sir-sigurd commented Nov 2, 2020

View reviewed changes

api/python/quilt3/workflows/config-1.schema.json Outdated Show resolved Hide resolved

sir-sigurd commented Nov 2, 2020

View reviewed changes

api/python/quilt3/workflows/config-1.schema.json Show resolved Hide resolved

sir-sigurd force-pushed the workflows branch 3 times, most recently from 66d32f3 to 3c2c3a4 Compare November 2, 2020 22:05

akarve reviewed Nov 3, 2020

View reviewed changes

api/python/quilt3/packages.py Show resolved Hide resolved

akarve reviewed Nov 3, 2020

View reviewed changes

docs/SUMMARY.md Show resolved Hide resolved

sir-sigurd force-pushed the workflows branch 6 times, most recently from a02b3f5 to da8036f Compare November 3, 2020 14:50

sir-sigurd added 7 commits November 4, 2020 18:23

[WIP] Metadata quality verification API

412aca7

fix docstring

1cb5939

import message for invalid config

13ba73d

polish tests

4d7f1d2

fix handling of invalid/unsupported URLs, more tests

db45191

polish tests

2a4606b

test handling of $schema

0c93941

sir-sigurd force-pushed the workflows branch from 4153868 to 0c93941 Compare November 4, 2020 13:23

sir-sigurd added 2 commits November 4, 2020 18:48

test supported meta-schema

7680772

update docstrings

3a2ddc8

sir-sigurd commented Nov 4, 2020

View reviewed changes

docs/API Reference/Package.md Show resolved Hide resolved

sir-sigurd requested a review from akarve November 4, 2020 18:48

sir-sigurd commented Nov 4, 2020

View reviewed changes

docs/Advanced Features/workflows.md Show resolved Hide resolved

sir-sigurd commented Nov 4, 2020

View reviewed changes

api/python/quilt3/workflows/config-1.schema.json Show resolved Hide resolved

sir-sigurd marked this pull request as ready for review November 5, 2020 09:28

akarve approved these changes Nov 10, 2020

View reviewed changes

sir-sigurd and others added 10 commits November 11, 2020 07:43

Apply suggestions from code review

d98fc17

Co-authored-by: Aneesh Karve <[email protected]>

fix exception message

ffdcddc

Merge branch 'master' into workflows

023cc12

adjust 'schemas' property description

56163cb

fix handling of schemas that are not valid JSON

ec205ed

add changelog entry

58db2a8

more tests

9ee9f97

Merge branch 'master' into workflows

e765232

more tests

a8abed1

remove unused import

637a31f

sir-sigurd changed the title ~~[WIP] Metadata quality verification API~~ Metadata quality verification API Nov 12, 2020

sir-sigurd requested a review from akarve November 12, 2020 16:41

sir-sigurd added 2 commits November 12, 2020 21:44

fix accidentally merged typo

2faea82

revert another accidental change

0e13da2

akarve approved these changes Nov 13, 2020

View reviewed changes

sir-sigurd and others added 7 commits November 13, 2020 09:27

Update api/python/quilt3/workflows/config-1.schema.json

282439c

Co-authored-by: Aneesh Karve <[email protected]>

handle error while loading schema

1dac449

test s3 schemas with versionId

586b8ef

remove unused import

0f0f89d

handle s3 errors while loading config

4d42f25

fix noop asserts

d2824d5

Merge branch 'master' into workflows

a02ee21

sir-sigurd merged commit 7bc5bab into master Nov 16, 2020

sir-sigurd deleted the workflows branch November 16, 2020 14:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Metadata quality verification API #1874

Metadata quality verification API #1874

sir-sigurd commented Oct 30, 2020 •

edited

Loading

codecov-io commented Oct 31, 2020 •

edited

Loading

akarve Nov 10, 2020

sir-sigurd Nov 11, 2020

akarve Nov 11, 2020

sir-sigurd Nov 12, 2020

akarve Nov 10, 2020

sir-sigurd Nov 12, 2020 •

edited

Loading

akarve Nov 13, 2020

akarve commented Nov 10, 2020

Metadata quality verification API #1874

Metadata quality verification API #1874

Conversation

sir-sigurd commented Oct 30, 2020 • edited Loading

Description

TODO

codecov-io commented Oct 31, 2020 • edited Loading

Codecov Report

akarve Nov 10, 2020

Choose a reason for hiding this comment

sir-sigurd Nov 11, 2020

Choose a reason for hiding this comment

akarve Nov 11, 2020

Choose a reason for hiding this comment

sir-sigurd Nov 12, 2020

Choose a reason for hiding this comment

akarve Nov 10, 2020

Choose a reason for hiding this comment

sir-sigurd Nov 12, 2020 • edited Loading

Choose a reason for hiding this comment

akarve Nov 13, 2020

Choose a reason for hiding this comment

akarve commented Nov 10, 2020

sir-sigurd commented Oct 30, 2020 •

edited

Loading

codecov-io commented Oct 31, 2020 •

edited

Loading

sir-sigurd Nov 12, 2020 •

edited

Loading