Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tail sampling] Review behaviour when Elasticsearch license checks fail #5125

Closed
Tracked by #6894
axw opened this issue Apr 20, 2021 · 2 comments · Fixed by #5952
Closed
Tracked by #6894

[tail sampling] Review behaviour when Elasticsearch license checks fail #5125

axw opened this issue Apr 20, 2021 · 2 comments · Fixed by #5952
Assignees
Labels
Milestone

Comments

@axw
Copy link
Member

axw commented Apr 20, 2021

Tail-based sampling currently requires a platinum license. We should ensure that when Elasticsearch is inaccessible, APM Server does not exit but instead retries requests later, particularly in case of intermittent network failures.

If the Elasticsearch request succeeds but the license is incompatible, then we set some kind of status indicator in Fleet, or reject the received Fleet policy outright.

@simitt simitt added the v7.15.0 label Jun 17, 2021
@zube zube bot added the [zube]: Ready label Jun 30, 2021
@zube zube bot added this to the 7.15 milestone Jul 1, 2021
@simitt simitt changed the title Review behaviour of tail-based sampling when Elasticsearch license checks fail [tail sampling] Review behaviour when Elasticsearch license checks fail Jul 1, 2021
@axw axw self-assigned this Jul 29, 2021
@axw
Copy link
Member Author

axw commented Jul 29, 2021

Running apm-server standalone with tail-based sampling enabled, the server exits immediately if Elasticsearch is unavailable as the server tries and fails to create a data stream for sampling decisions. Easily reproduced with:

./apm-server -e --strict.perms=false -E 'apm-server.sampling.tail.policies=[{sample_rate: 0.1}]'

@axw axw added v7.16.0 and removed v7.15.0 labels Aug 4, 2021
@axw axw modified the milestones: 7.15, 7.16 Aug 4, 2021
@axw axw modified the milestones: 7.16, 7.15 Aug 16, 2021
@axw axw added v7.15.0 and removed v7.16.0 labels Aug 16, 2021
@axw
Copy link
Member Author

axw commented Aug 16, 2021

This will be fixed by #5952, which removes the data stream creation from standalone apm-server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants