Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: caching: fix early termination expression check & cabal.project replacement #2520

Merged
merged 9 commits into from
Dec 22, 2021

Conversation

Anton-Latukha
Copy link
Collaborator

@Anton-Latukha Anton-Latukha commented Dec 21, 2021

Details are in the commit messages.

  1. We not noticed the first specific of the CI, because the early cutoff behaved as it should. It was respected to return 'false' to terminate early, but that check always returned 'false'. When working in GHC 9.2 PR could not force the caching build to start, because check that I fix - always skipped the steps (always returned False). New check is done in the way how it is done in other if: ... != 'true' statements in the workflow.

  2. The second specific is both Ubuntu-specific & maybe GitHub specific. Some OSes run alias cp='cp -i' & when I tried to rm manually - cabal.project appeared to be protected from simple rm by setinning (most probably) of a "sticky bit" on $HOME directory. I found that-out by working in GHC 9.2 PR (where index-state is actually different between .project files & noticed that CI cache key was with the same inferred index-state in all jobs). Now rm -f forced & that allows removing file with a sticky bit. log reports the situation of files changes there: https://github.com/haskell/haskell-language-server/runs/4597911492?check_suite_focus=true#step:14:460

@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented Dec 21, 2021

Ok. Now lets actually check the early termination & after that the new key (should go into full run).

@Anton-Latukha
Copy link
Collaborator Author

It is obscure bug.

Details are in
https://matrix.to/#/!oOjZFsoNYPAbTEgSOA:libera.chat/$tB_L6wbaxe1ElQtYzMI9woID4hZyscS6Rhpn-nsCc6Y?via=libera.chat&via=matrix.org&via=monoid.al

Because after the second path somehow it always returned `False` - the caching
runs always resulted in early termination.
In the
https://github.com/haskell/haskell-language-server/runs/4596861600?check_suite_focus=true
of
haskell#2503

Noted that:

I rm cabal.project. The sources of cabal-ghc921.project contain:
index-state: 2021-12-18T00:00:07Z
cp cabal-ghc921.project cabal.project
Then from cabal.project CI retrieves finderprint 2021-11-29T08:11:08Z.
Seems like GitHub uses alias cp=cp -i - preventing UNIX-like default copying over behaviour.
& rm cabal.project does not remove the file.
cabal.project seems to be protected from deletion.
@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented Dec 21, 2021

I advanced the index-state.

Which changes the key. So the old key of cache & new id for cache of the workflow does not align & so old storage is loaded, but the full build&cache process starts & then saves into the new key.

Resulting into the proper long caching run: https://github.com/haskell/haskell-language-server/runs/4599160496?check_suite_focus=true

@Anton-Latukha
Copy link
Collaborator Author

Ok, now I need to wait until this run completes & then drop the modifications. & have a 1 clean run on {test,bench}, it is honest to do at least because I changed test workflow.

@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented Dec 21, 2021

Ok, now run is complete https://github.com/haskell/haskell-language-server/pull/2520/checks.

By dropping the WIP commit - disabling caching & enabling {test,bench}.

@Anton-Latukha
Copy link
Collaborator Author

@Anton-Latukha Anton-Latukha marked this pull request as ready for review December 21, 2021 21:32
@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented Dec 21, 2021

Seems I checked everything about early termination this time.

& seem that cabal.project switcharoo now works.

I hope we solved the main problems there & hope that soon we would exhaust hidden from us bugs in the CI to become sure in configuration robustness.

Copy link
Member

@jneira jneira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, good catches, thanks

@jneira jneira added the merge me Label to trigger pull request merge label Dec 22, 2021
@mergify mergify bot merged commit e6c1fca into haskell:master Dec 22, 2021
@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented Dec 22, 2021

Would note, that now because according .project are loaded & tested under - CI pipeline would ship according context & so uncover some related to that effects/bugs inside pull request processes.

@Anton-Latukha Anton-Latukha deleted the 2021-12-21-ci-fxs branch December 22, 2021 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge me Label to trigger pull request merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants