Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding validation checks for Bookkeeper Deployments #598

Merged
merged 2 commits into from
Dec 7, 2021

Conversation

nishant-yt
Copy link
Collaborator

Signed-off-by: Nishant Gupta [email protected]

Change log description

When deploying the pravega cluster, the options bookkeeper.ensemble.size bookkeeper.write.quorum.size bookkeeper.ack.quorum.size should be set such that the following condition is satisfied: bookkeeper.ensemble.size >= bookkeeper.write.quorum.size >= bookkeeper.ack.quorum.size

Purpose of the change

Fixes #587

What the code does

The code preforms the following checks :

  • bookkeeper.ensemble.size >= bookkeeper.write.quorum.size >= bookkeeper.ack.quorum.size
  • If bookkeeper.ensemble.size == 1 then bookkeeper.write.quorum.racks.minimumCount.enable should be set to false

How to verify it

Install a pravega cluster with different values of bookkeeper.ensemble.size bookkeeper.write.quorum.size bookkeeper.ack.quorum.size and in case the above mentioned condition in not met , then an error is retuned by the validating webhook.
In case the values adheres to the above condition say bookkeeper.ensemble.size=4 bookkeeper.write.quorum.size=3 bookkeeper.ack.quorum.size=3 then post installation of the pravega cluster if you go inside Controller/ Segment Store Pod then all these values should be available in the env variable JAVA_OPTS as

JAVA_OPTS=-Dbookkeeper.ack.quorum.size=3 -Dbookkeeper.ensemble.size=4 -Dbookkeeper.write.outstanding.bytes.max=33554432 -Dbookkeeper.write.quorum.size=3 -Dbookkeeper.write.timeout.milliseconds=60000 -Dcontroller.container.count=8 -Dcontroller.retention.bucket.count=4 -Dcontroller.retention.thread.count=4 -Dcontroller.service.asyncTaskPool.size=20

@codecov-commenter
Copy link

codecov-commenter commented Dec 3, 2021

Codecov Report

Merging #598 (1823530) into master (e7af6d1) will increase coverage by 0.15%.
The diff coverage is 88.23%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #598      +/-   ##
==========================================
+ Coverage   74.88%   75.03%   +0.15%     
==========================================
  Files          16       16              
  Lines        4428     4479      +51     
==========================================
+ Hits         3316     3361      +45     
- Misses        978      984       +6     
  Partials      134      134              
Impacted Files Coverage Δ
pkg/apis/pravega/v1beta1/pravegacluster_types.go 33.11% <85.71%> (+2.50%) ⬆️
pkg/apis/pravega/v1beta1/pravega.go 98.88% <100.00%> (+0.05%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7af6d1...1823530. Read the comment docs.


if ensembleSizeInt < writeQuorumSizeInt {
if ensembleSize == "" {
return fmt.Errorf("The value provided for the option bookkeeper.write.quorum.size should be less than or equal to the default value of option bookkeeper.ensemble.size which is 3")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The message is a bit confusing (the user might have forgotten to set bookkeeper.ensemble.size altogether).
Maybe change it to something like this:
"The value provided for the option bookkeeper.write.quorum.size should be less than or equal to the value of option bookkeeper.ensemble.size (default is 3)"
Then the user would be aware of the need to either set the ensemble size explicitly or coordinate the quorum size with the default.

if ackQuorumSize == "" {
return fmt.Errorf("The value provided for the option bookkeeper.write.quorum.size should be greater than or equal to the default value of bookkeeper.ack.quorum.size which is 3")
}
return fmt.Errorf("The value provided for the option bookkeeper.ack.quorum.size should less than or equal to the value of option bookkeeper.write.quorum.size")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) should less than -> should be less than

Signed-off-by: Nishant Gupta <[email protected]>
@@ -383,6 +383,21 @@ func (s *PravegaSpec) withDefaults() (changed bool) {
s.Options = map[string]string{}
}

if s.Options["bookkeeper.ensemble.size"] == "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nishant-yt since we are making all the 3 fields mandatory, will it enter this code flow

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anishakj all the fields are not mandatory. If the user doesn't provide value for any of these options, we are setting it to 3 by default.
One of instances when it will enter the above mentioned code flow is when the user doesn't provide any value for bookkeeper.ensemble.size bookkeeper.write.quorum.size and bookkeeper.ack.quorum.size

Copy link
Contributor

@anishakj anishakj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@anishakj anishakj merged commit ff535f8 into master Dec 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adding a values check for 1-BK deployments
4 participants