Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add regression-tests.yml for manually testing dependent packages #2323

Merged
merged 1 commit into from
Nov 2, 2023

Conversation

odow
Copy link
Collaborator

@odow odow commented Nov 1, 2023

Closes #2314

I don't know if I got this right, so we'll need to test.

Running the dependent package tests is easy. Checking that we can build other package docs is...less easy. For example, JuMP currently pins the version of Documenter, https://github.com/jump-dev/JuMP.jl/blob/908275f105d7902d0065aad8a56a2caf80e391cf/docs/Project.toml#L43, so trying to build here will fail.

If we knew there were regression tests in Documenter we might relax that though.

@odow
Copy link
Collaborator Author

odow commented Nov 2, 2023

Now I remember: this is a problem with GitHub. You can only kick off a workflow-dispatch once the workflow has been added. and then once it has, you can create a branch, edit, open a PR, and then run it against the branch.

So the only way to tell if this works is to merge it...which isn't a big deal, because it can only be run manually.

@mortenpi
Copy link
Member

mortenpi commented Nov 2, 2023

One quick thing to double check: the code running in this workflow doesn't have access to user secrets, right? We would have to explicitly pass them to the jobs via env? We're running arbitrary user code here, so just want to be extra sure.

@odow
Copy link
Collaborator Author

odow commented Nov 2, 2023

It only has access to the normal environment variables, which shouldn't be sensitive. But it's something to be mindful of. You probably don't want to try and run arbitrary packages.

When I've needed to, I've isolated packages into their own job and given them access to a secret:
https://github.com/jump-dev/MathOptInterface.jl/blob/b4398223041e4427d9122cbb971648bffe6b80d3/.github/workflows/solver-tests.yml#L77-L112

@mortenpi
Copy link
Member

mortenpi commented Nov 2, 2023

Yeah, but I don't think the regression tests need access to secrets anyway.

Should we fix the packages to specific versions/commits though?

Copy link
Member

@mortenpi mortenpi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, feel free to merge this, if you want to trigger this for testing.

@odow
Copy link
Collaborator Author

odow commented Nov 2, 2023

Should we fix the packages to specific versions/commits though?

Nah. We typically just pull in the latest master. That assumes master is always good, and supports the latest Documenter. But this is a sanity check anyway, so the odd failure doesn't matter. It just prods you to go and take a look at the failure.

@odow odow merged commit c6abf9d into master Nov 2, 2023
21 checks passed
@odow odow deleted the od/regression-tests branch November 2, 2023 04:05
@odow
Copy link
Collaborator Author

odow commented Nov 2, 2023

This seems to have worked https://github.com/JuliaDocs/Documenter.jl/actions/runs/6728161571

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GitHub Actions workflows for downstream testing
2 participants