fix: building withing lower parallelism than dependency group size #2051
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2042
Before this change, we would get build errors because the
go.work
file for a given module would include all the built modules including modules that were in the same buildgroup
layer as a given module.For example, if we had 4 modules all in the same group
[a, b, c, d]
and we build with a-j2
option (2 parallel builds), we would run into an issue wherea
andb
would build first, in parallel.a
would finish and we would start to buildc
. But, when we looked up theschema
to get shared modules for thego.work
file forc
, that list would now includea
even though they are in the same build group and should never depend on each other.This fix changes the build func to not re-lookup built modules and instead just use the set of
builtModules
that were available at the start of the build group.With this change we should be able to safely support any
-j
option higher or lower than the number of modules. This probably would have eventually happened either way once we had a system with more modules than the number of cpus on a given machine.