Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CICD schema and render workflows #1068

Merged
merged 17 commits into from
Feb 17, 2022
Merged

Add CICD schema and render workflows #1068

merged 17 commits into from
Feb 17, 2022

Conversation

iameskild
Copy link
Member

Fixes | Closes | Resolves #1013

Please remove anything marked as optional that you don't need to fill in. Choose one of the keywords preceding to refer to the issue this PR solves, followed by the issue number (e.g Fixes # 666). If there is no issue, remove the line. Remove this note after reading.

Changes:

Much of this code was inspired by our existing schema.py and from conversations and discussion had with @costrouc and @tonyfast. Thank you both for your assistance and feedback.

  • Create GitHub-Action workflow schema
  • Create workflows based on schema

Types of changes

What types of changes does your code introduce?

Put an x in the boxes that apply

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds a feature)
  • Breaking change (fix or feature that would cause existing features to not work as expected)
  • Documentation Update
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no API changes)
  • Build related changes
  • Other (please describe):

Testing

Requires testing

  • Yes
  • No

In case you checked yes, did you write tests?

  • Yes
  • No

Not yet...

Further comments (optional)

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered and more.

@iameskild
Copy link
Member Author

iameskild commented Feb 16, 2022

@costrouc this is what I have so far, I still have a few things to wrap up but I'm happy with this outline so far. Let me know if you have questions, concerns or suggestions :)

Outstanding:

  • Create GitLab-CI workflow schema
  • Create qhub-ops workflow for GitLab-CI
  • End-to-end testing

@iameskild
Copy link
Member Author

@costrouc workflow files being deprecated:

  • image-pr.yaml
  • image.yaml
  • jupyterhub-pr.yaml
  • jupyterhub.yaml

@costrouc
Copy link
Member

costrouc commented Feb 17, 2022

Originally I was going to say we should convert this to just yaml like syntax instead of sticking with json which can be interpreted as yaml. But the json is extremely readable and I think we should stick with it since it is less ambiguous.

I do see the future concern tha the QHub Provider check produces (3 x 5) checks. We may in the future want to combine these. But for now that is totally okay.

@costrouc
Copy link
Member

I also see the version issue on pip install qhub==... which I think is only showing itself with a development version.

@iameskild
Copy link
Member Author

@costrouc keeping it in json is fine with me!
Thinking through pip install qhub==... version issue, this might make it a little tricky to test since the deployment version of qhub won't be on pypi. Perhaps we can add some logic to see if the version of qhub is "dirty" and if so install from github otherwise from pypi.

@costrouc
Copy link
Member

I'm fine for now just merging this work and leaving it to another PR to figure this out for development. For an actual Pypi release this won't be a problem

Copy link
Contributor

@viniciusdc viniciusdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing, the fact that we are using schema for it makes a good difference when testing in the future. Amazing work @iameskild, have you tested the gitlab actions?

branch = config["ci_cd"]["branch"]
qhub_version = config["qhub_version"]

push = GHA_on_extras(branches=[branch], path=["qhub-config.yaml"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we considering #995 here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see this but I think won't be too hard. Do you think we should include it here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, as it was linked to the original issue and it would be just adding an if for step5 based on this new variable

@@ -349,3 +349,15 @@ def deep_merge(*args):
return [*d1, *d2]
else: # if they don't match use left one
return d1


def pip_install_qhub(qhub_version: str) -> str:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch 👍

Copy link
Member Author

@iameskild iameskild Feb 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@costrouc this is an unsophisticated way of handling the qhub_version issue for now. Would this be okay to add?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeup this works with me.

Copy link
Member

@costrouc costrouc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iameskild looks great to me. After merging this we can test this and impove upon it when we find issues.

@costrouc costrouc merged commit a62a321 into main Feb 17, 2022
@costrouc costrouc deleted the feat-1013-cicd branch February 17, 2022 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add CI/CD pipelines for github and gitlab back
3 participants