Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure to read association configuration again from annotations if it was cleared #5489

Merged
merged 14 commits into from
Mar 24, 2022

Conversation

thbkrkr
Copy link
Contributor

@thbkrkr thbkrkr commented Mar 16, 2022

This commit introduces two new functions GetAndSetAssociationConf()and GetAndSetAssociationConfByRef() to be called in the implementation of AssociationConf() for each Association to be sure that if there is an update of the associated resource that carries the association that resets the assocConf, then we make sure to read it back from the annotation.

This should be called very rarely because we re-read the annotation only if the current assoc conf is not set.

Due to a circular dependency, I move the functions in the pkg/apis/common/v1 package.

Note: I couldn't reproduce the issue that resets non persisted attributes during an update with the fakeClient.

Resolves #4709.

@thbkrkr thbkrkr added >bug Something isn't working v2.2.0 labels Mar 16, 2022
}
assert.Equal(t, 2, len(esMon.AssocConfs))

// simulate the case where the assocConfs map is reset, which can happen if the resource is updated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simulate the case where the assocConfs map is reset, which can happen if the resource is updated

Sorry, I'm not sure to understand. assocConfs is not persisted, I think it is expected for it to be empty before FetchWithAssociations kicks in ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue happens after FetchWithAssociations, when the resource is updated, it resets the map. This test shows that if it is reset after SetAssociationConf(), the map is repopulated on the fly by AssociationConf().
Maybe I should go further and delete SetAssociationConf() to avoid confusion. We could even delete the FetchWithAssociations() and just rely on AssociationConf() called in different places.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could even delete the FetchWithAssociations() and just rely on AssociationConf() called in different places.

Thanks for the clarification ! My understanding is that it means that FetchWithAssociations is unreliable 😕 I wonder if it's really worth keeping it along with the unserialized annotations ? IIUC they are only here to spare a few CPU cycles by not using json.Unmarshal when the association config. is needed ?

Copy link
Contributor

@barkbay barkbay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Due to a circular dependency, I created a new utils package under association

Could we move these new functions in pkg/apis/common/v1 ? It would avoid the addition of a new package and solve the circular dependency.

pkg/controller/association/utils/utils.go Outdated Show resolved Hide resolved
pkg/controller/association/utils/utils.go Outdated Show resolved Hide resolved
@thbkrkr thbkrkr requested a review from barkbay March 23, 2022 15:16
Copy link
Contributor

@barkbay barkbay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pkg/apis/common/v1/conf.go Outdated Show resolved Hide resolved
pkg/apis/common/v1/conf.go Outdated Show resolved Hide resolved
pkg/apis/common/v1/conf.go Outdated Show resolved Hide resolved
pkg/apis/common/v1/conf.go Outdated Show resolved Hide resolved
pkg/controller/apmserver/controller.go Outdated Show resolved Hide resolved
@barkbay
Copy link
Contributor

barkbay commented Mar 24, 2022

As I already mentioned I still have the feeling that these "non-persisted" annotations bring more complexity than value, but overall I think I'm fine keeping them.

@thbkrkr
Copy link
Contributor Author

thbkrkr commented Mar 24, 2022

As this is already in place and is now well isolated and reliable, I am in favor of keeping it.

@thbkrkr thbkrkr merged commit 1970e96 into elastic:main Mar 24, 2022
@thbkrkr thbkrkr changed the title Make sure to read association configuration again from annotations if it was reset Make sure to read association configuration again from annotations if it was cleared Mar 24, 2022
@barkbay
Copy link
Contributor

barkbay commented Mar 28, 2022

@thbkrkr : just to double check, does it really resolve #4709 ? I'm asking because that issue is linked to this other one: #4627

@thbkrkr
Copy link
Contributor Author

thbkrkr commented Mar 28, 2022

@thbkrkr : just to double check, does it really resolve #4709 ? I'm asking because that issue is linked to this other one: #4627

Yes it fixes #4709 and so #4627. Version upgrade for ES cluster with self monitoring was fixed by #5339.

@thbkrkr thbkrkr deleted the fix-assoc-conf branch May 17, 2022 09:50
fantapsody pushed a commit to fantapsody/cloud-on-k8s that referenced this pull request Feb 7, 2023
… it was reset (elastic#5489)

This commit introduces two new functions `GetAndSetAssociationConf()` and `GetAndSetAssociationConfByRef()` to be called in the implementation of `AssociationConf()` for each Association to be sure that if there is an update of the associated resource that carries the association that clears the `assocConf` map, then we make sure to read it back from the annotation.

This fix makes the `assocConf` map reliable and has also the nice side-effect to avoid an unnecessary initial restart when deploying a self-monitoring Elasticsearch cluster.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug Something isn't working v2.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot use Stack Monitoring to have Elasticsearch monitor itself
2 participants