Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Cirun on self-hosted GHA runners #1703

Merged
merged 75 commits into from
Nov 6, 2023

Conversation

jaimergp
Copy link
Member

@jaimergp jaimergp commented Dec 19, 2022

@isuruf
Copy link
Member

isuruf commented Oct 23, 2023

It's a bug from #1752

conda_smithy/cirun_utils.py Outdated Show resolved Hide resolved
conda_smithy/cirun_utils.py Outdated Show resolved Hide resolved
conda_smithy/cirun_utils.py Outdated Show resolved Hide resolved
conda_smithy/cirun_utils.py Outdated Show resolved Hide resolved
@isuruf isuruf marked this pull request as ready for review October 31, 2023 13:58
@isuruf isuruf requested a review from a team as a code owner October 31, 2023 13:58
@beckermr
Copy link
Member

I don't think I have expertise to review this PR. However, can we make the docs PR sooner rather than later? I think that will help others figure out what to do with all of these new features.

@isuruf isuruf changed the title (WIP) Add support for Cirun on self-hosted GHA runners Add support for Cirun on self-hosted GHA runners Oct 31, 2023
@isuruf
Copy link
Member

isuruf commented Oct 31, 2023

Docs PR: conda-forge/conda-forge.github.io#2025

@isuruf
Copy link
Member

isuruf commented Nov 2, 2023

@conda-forge/core, ready for a review

@ocefpaf
Copy link
Member

ocefpaf commented Nov 6, 2023

I don't think I have expertise to review this PR.

Same here but we can take a pragmatic approach by answering the following questions:

  • Does this break existing code? Doesn't seem like it, so +1 to merge.
  • Does this leak anything that shouldn't? Also doesn't seem like it, another OK to merge.

If both my answers above make sense then we can ask: Does it work for those testing it? Or do you need this merge to put it in production? If so, let's merge.

However, can we make the docs PR sooner rather than later? I think that will help others figure out what to do with all of these new features.

+100

@jaimergp
Copy link
Member Author

jaimergp commented Nov 6, 2023

Basic docs have been merged. There's a chance we need to write more, but for now I think they are sufficient to get us unblocked here and merge to try things.

conda_smithy/cirun_utils.py Show resolved Hide resolved
@@ -2008,6 +2091,17 @@ def _load_forge_config(forge_dir, exclusive_config_file, forge_yml=None):
if config["test"] is None:
config["test"] = "all"

if config["github_actions"]["cancel_in_progress"] is None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reasoning of this? Why bind cancel_in_progress to whether the runners are self-hosted?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To save CI resources by default. Self-hosted runners are usually going to be scarce or costly. If a user wants to not cancel-in-progress, they can explicitly say so, but by default it will cancel previous jobs when self-hosted runners are in use.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also cancel hosted ones, too. They are scarce (only a limited pool of them) and can also be costly if you are not using your free minutes (or a runner that falls into the free minutes).

I would use the rather simple option of

if config["github_actions"]["cancel_in_progress"] is None:
    config["github_actions"]["cancel_in_progress"] = True

Probably you can just set the default to False?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the cancel in progress here is configured to cancel all jobs in the same build. For eg: Linux builds will get cancelled if windows builds fail.

@isuruf isuruf merged commit b1c5fca into conda-forge:main Nov 6, 2023
2 checks passed
@@ -695,7 +698,6 @@ def _render_ci_provider(
channel_target.startswith("conda-forge ")
and provider_name == "github_actions"
and not forge_config["github_actions"]["self_hosted"]
and os.path.basename(forge_dir) not in SERVICE_FEEDSTOCKS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaimergp why was this line removed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want it to be self-hosted or in service

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants