Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel/ci: Improve flags/config #28856

Merged
merged 1 commit into from
Aug 7, 2023
Merged

Conversation

phlax
Copy link
Member

@phlax phlax commented Aug 5, 2023

Currently our bazel flags are not very well structured

  • lots of duplicate flags (-> indeterminate outcomes)
  • not well setup for both rbe <> local
  • not such good separation of responsibility

the problem is 10x worse in mobile, which also hijacks quite a few flags from envoys config

this rinses out duplicates in the config, separates mobile config to its own ~namespace and gets rid of related warnings

this also quietens mobile CI and standardizes the RBE config which will allow adding more complex setups (#28814 )

also removes github mobile token condition that would never be hit and tightens workflow permissions generally

also separates github workflow diskspace cleanup code to an action

and switches default (mostly) for remote exec to --remote_download_minimal, updating any ci that requires more downloads, runfiles, etc

more generally moves/consolidates config to the .bazelrc file to make it easier to manage them

broadly, its some first steps to addressing the issues posted above in terms of separating responsibility

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

@phlax phlax marked this pull request as draft August 5, 2023 16:02
@phlax phlax force-pushed the bazel-flags-rbe branch 4 times, most recently from 347e6be to 7d4b193 Compare August 5, 2023 19:55
.bazelrc Outdated Show resolved Hide resolved
@phlax phlax force-pushed the bazel-flags-rbe branch 2 times, most recently from bf4251c to 7351e87 Compare August 5, 2023 20:38
@phlax phlax changed the title [WIP] bazel/ci: Improve bazel remote flags [WIP] bazel/ci: Improve bazel flags/config Aug 5, 2023
@phlax phlax changed the title [WIP] bazel/ci: Improve bazel flags/config [WIP] bazel/ci: Improve flags/config Aug 5, 2023
@phlax phlax force-pushed the bazel-flags-rbe branch 9 times, most recently from d087630 to e60ba77 Compare August 6, 2023 09:36
@phlax phlax force-pushed the bazel-flags-rbe branch 2 times, most recently from 496d7e7 to d6c7afd Compare August 6, 2023 09:56
ci/do_ci.sh Outdated Show resolved Hide resolved
@phlax phlax force-pushed the bazel-flags-rbe branch 4 times, most recently from 6b5da9b to 16d6b0a Compare August 6, 2023 12:52
@phlax phlax force-pushed the bazel-flags-rbe branch 4 times, most recently from 4a81645 to 13bbd4d Compare August 6, 2023 16:23
@phlax phlax changed the title [WIP] bazel/ci: Improve flags/config bazel/ci: Improve flags/config Aug 6, 2023
@phlax phlax marked this pull request as ready for review August 6, 2023 16:31
@phlax phlax force-pushed the bazel-flags-rbe branch from 13bbd4d to 3ed38ce Compare August 6, 2023 16:54
@phlax
Copy link
Member Author

phlax commented Aug 6, 2023

this PR is quite invasive but has been extensively tested, fixes a stack of problems and lays the groundwork to fix a stack more

@htuch
Copy link
Member

htuch commented Aug 6, 2023

Sending this to @RyanTheOptimist @keith due to the Envoy Mobile implications.

@RyanTheOptimist
Copy link
Contributor

Does this impact the current ability of failed mobile job to be restarted (which doesn't seem to be available for non-mobile jobs)

@phlax
Copy link
Member Author

phlax commented Aug 7, 2023

Does this impact the current ability of failed mobile job to be restarted (which doesn't seem to be available for non-mobile jobs)

not at all - i havent touched the job handling at all in this PR

Copy link
Contributor

@RyanTheOptimist RyanTheOptimist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this LGTM. There are quite a few changes, and this code/config can be a bit subtle, but overall it looks like a really nice cleanup.

@@ -40,7 +40,7 @@ Do not allow any bots or app users to do so, unless this is specifically require
For example, you could add a `job` condition to prevent any bots from triggering the workflow:

```yaml
if: |
if: >-
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OOC, what does >- mean here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strip ws - previously this condition was always returning true as false ~= " "

@phlax
Copy link
Member Author

phlax commented Aug 7, 2023

There are quite a few changes, and this code/config can be a bit subtle

thanks - there are a few things (eg re configs) that should hopefully become clearer with subsequent prs - i will monitor this closely going through, and have some behind to rebase and push forward also

@phlax phlax merged commit 4580338 into envoyproxy:main Aug 7, 2023
phlax added a commit to phlax/envoy that referenced this pull request Aug 8, 2023
phlax added a commit to phlax/envoy that referenced this pull request Aug 8, 2023
@phlax phlax mentioned this pull request Aug 9, 2023
abeyad added a commit to abeyad/envoy that referenced this pull request Aug 14, 2023
The mobile Bazel options were updated in
envoyproxy#28856, but the mobile_release
workflow wasn't updated to use mobile-release-android. It was probably
not caught by CI because it is a manual workflow that is run once a week
as opposed to running on every PR.

Signed-off-by: Ali Beyad <[email protected]>
abeyad added a commit to abeyad/envoy that referenced this pull request Aug 14, 2023
The mobile Bazel options were updated in
envoyproxy#28856, but the mobile_release
workflow wasn't updated to use mobile-release-android. It was probably
not caught by CI because it is a manual workflow that is run once a week
as opposed to running on every PR.

Signed-off-by: Ali Beyad <[email protected]>
abeyad added a commit that referenced this pull request Aug 14, 2023
#29009)

The mobile Bazel options were updated in
#28856, but the mobile_release
workflow wasn't updated to use mobile-release-android. It was probably
not caught by CI because it is a scheduled workflow that is run once a week
as opposed to running on every PR.

Signed-off-by: Ali Beyad <[email protected]>
phlax added a commit that referenced this pull request Aug 21, 2023
Signed-off-by: Ryan Northey <[email protected]>

Signed-off-by: phlax <[email protected]>
phlax added a commit that referenced this pull request Aug 22, 2023
Signed-off-by: Ryan Northey <[email protected]>

Signed-off-by: phlax <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants