[Profiling] Add APIs to clear frame and executable caches #145499
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The two new endpoints will clear each respective cache.
jbcrail
added
release_note:skip
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Page load bundle
Unknown metric groupsESLint disabled in files
ESLint disabled line counts
Total ESLint disabled count
To update your PR or re-run it, just comment with: |
|
@jbcrail What happens if multiple kibana instances are running? Is there a way to clear the cache for all instances? |
@dgieselaar How does a client find out about multiple Kibana instances ? If that is possible, the client could take care of that, in case Kibana instances doesn't know of each other. |
|
@rockdaboot It's fairly simple, multiple kibana instances in a cluster can handle a request, via presumably something similar to round-robin. Cache will be local to the kibana instance, so the user has to call this endpoint multiple times in the case of multiple kibana instances. Two things we can possibly improve here:
|
Thanks Dario. |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Nov 17, 2022
…5499) This PR adds two endpoints to clear each respective cache: * `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables` * `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes` Related to https://github.com/elastic/prodfiler/issues/2759 #### Design choices 1. The `DELETE` method was chosen instead of `PUT` or `POST` since the given semantics of `DELETE` matches the expected behavior for the related issue. 2. Each endpoint will remove all items from the respective cache. A separate API for each cache allows us to selectively clear the necessary cache without the downsides of a catch-all endpoint to clear all caches. This gives us the flexibility to add more endpoints if needed. Given the tradeoff between complexity now or later, it was decided to implement general invalidation now with the option to invalidate specific items later. 3. The RESTful design allows us to clear specific items later (e.g. `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}` could clear only executable `ID` from the cache). 4. Each endpoint returns an empty payload on success. However, the Kibana logs reflect the actions taken and how many cache items were affected. 5. The stacktrace cache was ignored since it was not affected by symbols written to Elasticsearch. 6. Each endpoint is not directly accessible from UI since it is expected that the endpoints will be called outside of Kibana. However, the endpoints can be called manually from the browser's console. (cherry picked from commit 642ab74)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Nov 17, 2022
…5499) This PR adds two endpoints to clear each respective cache: * `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables` * `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes` Related to https://github.com/elastic/prodfiler/issues/2759 #### Design choices 1. The `DELETE` method was chosen instead of `PUT` or `POST` since the given semantics of `DELETE` matches the expected behavior for the related issue. 2. Each endpoint will remove all items from the respective cache. A separate API for each cache allows us to selectively clear the necessary cache without the downsides of a catch-all endpoint to clear all caches. This gives us the flexibility to add more endpoints if needed. Given the tradeoff between complexity now or later, it was decided to implement general invalidation now with the option to invalidate specific items later. 3. The RESTful design allows us to clear specific items later (e.g. `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}` could clear only executable `ID` from the cache). 4. Each endpoint returns an empty payload on success. However, the Kibana logs reflect the actions taken and how many cache items were affected. 5. The stacktrace cache was ignored since it was not affected by symbols written to Elasticsearch. 6. Each endpoint is not directly accessible from UI since it is expected that the endpoints will be called outside of Kibana. However, the endpoints can be called manually from the browser's console. (cherry picked from commit 642ab74)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Nov 17, 2022
) (#145525) # Backport This will backport the following commits from `main` to `8.6`: - [[Profiling] Add APIs to clear frame and executable caches (#145499)](#145499) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Joseph Crail","email":"[email protected]"},"sourceCommit":{"committedDate":"2022-11-17T10:04:58Z","message":"[Profiling] Add APIs to clear frame and executable caches (#145499)\n\nThis PR adds two endpoints to clear each respective cache:\r\n\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables`\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes`\r\n\r\nRelated to https://github.com/elastic/prodfiler/issues/2759\r\n\r\n#### Design choices\r\n\r\n1. The `DELETE` method was chosen instead of `PUT` or `POST` since the\r\ngiven semantics of `DELETE` matches the expected behavior for the\r\nrelated issue.\r\n2. Each endpoint will remove all items from the respective cache. A\r\nseparate API for each cache allows us to selectively clear the necessary\r\ncache without the downsides of a catch-all endpoint to clear all caches.\r\nThis gives us the flexibility to add more endpoints if needed. Given the\r\ntradeoff between complexity now or later, it was decided to implement\r\ngeneral invalidation now with the option to invalidate specific items\r\nlater.\r\n3. The RESTful design allows us to clear specific items later (e.g.\r\n`DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}`\r\ncould clear only executable `ID` from the cache).\r\n4. Each endpoint returns an empty payload on success. However, the\r\nKibana logs reflect the actions taken and how many cache items were\r\naffected.\r\n5. The stacktrace cache was ignored since it was not affected by symbols\r\nwritten to Elasticsearch.\r\n6. Each endpoint is not directly accessible from UI since it is expected\r\nthat the endpoints will be called outside of Kibana. However, the\r\nendpoints can be called manually from the browser's console.","sha":"642ab74cfc1a4ef5c0a2dfdadf00bd3792f780a7","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.6.0","v8.7.0","v8.5.2"],"number":145499,"url":"https://github.com/elastic/kibana/pull/145499","mergeCommit":{"message":"[Profiling] Add APIs to clear frame and executable caches (#145499)\n\nThis PR adds two endpoints to clear each respective cache:\r\n\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables`\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes`\r\n\r\nRelated to https://github.com/elastic/prodfiler/issues/2759\r\n\r\n#### Design choices\r\n\r\n1. The `DELETE` method was chosen instead of `PUT` or `POST` since the\r\ngiven semantics of `DELETE` matches the expected behavior for the\r\nrelated issue.\r\n2. Each endpoint will remove all items from the respective cache. A\r\nseparate API for each cache allows us to selectively clear the necessary\r\ncache without the downsides of a catch-all endpoint to clear all caches.\r\nThis gives us the flexibility to add more endpoints if needed. Given the\r\ntradeoff between complexity now or later, it was decided to implement\r\ngeneral invalidation now with the option to invalidate specific items\r\nlater.\r\n3. The RESTful design allows us to clear specific items later (e.g.\r\n`DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}`\r\ncould clear only executable `ID` from the cache).\r\n4. Each endpoint returns an empty payload on success. However, the\r\nKibana logs reflect the actions taken and how many cache items were\r\naffected.\r\n5. The stacktrace cache was ignored since it was not affected by symbols\r\nwritten to Elasticsearch.\r\n6. Each endpoint is not directly accessible from UI since it is expected\r\nthat the endpoints will be called outside of Kibana. However, the\r\nendpoints can be called manually from the browser's console.","sha":"642ab74cfc1a4ef5c0a2dfdadf00bd3792f780a7"}},"sourceBranch":"main","suggestedTargetBranches":["8.6","8.5"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/145499","number":145499,"mergeCommit":{"message":"[Profiling] Add APIs to clear frame and executable caches (#145499)\n\nThis PR adds two endpoints to clear each respective cache:\r\n\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables`\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes`\r\n\r\nRelated to https://github.com/elastic/prodfiler/issues/2759\r\n\r\n#### Design choices\r\n\r\n1. The `DELETE` method was chosen instead of `PUT` or `POST` since the\r\ngiven semantics of `DELETE` matches the expected behavior for the\r\nrelated issue.\r\n2. Each endpoint will remove all items from the respective cache. A\r\nseparate API for each cache allows us to selectively clear the necessary\r\ncache without the downsides of a catch-all endpoint to clear all caches.\r\nThis gives us the flexibility to add more endpoints if needed. Given the\r\ntradeoff between complexity now or later, it was decided to implement\r\ngeneral invalidation now with the option to invalidate specific items\r\nlater.\r\n3. The RESTful design allows us to clear specific items later (e.g.\r\n`DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}`\r\ncould clear only executable `ID` from the cache).\r\n4. Each endpoint returns an empty payload on success. However, the\r\nKibana logs reflect the actions taken and how many cache items were\r\naffected.\r\n5. The stacktrace cache was ignored since it was not affected by symbols\r\nwritten to Elasticsearch.\r\n6. Each endpoint is not directly accessible from UI since it is expected\r\nthat the endpoints will be called outside of Kibana. However, the\r\nendpoints can be called manually from the browser's console.","sha":"642ab74cfc1a4ef5c0a2dfdadf00bd3792f780a7"}},{"branch":"8.5","label":"v8.5.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Joseph Crail <[email protected]>
kibanamachine
added
the
backport missing
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
kibanamachine
added a commit
that referenced
this pull request
Nov 18, 2022
) (#145524) # Backport This will backport the following commits from `main` to `8.5`: - [[Profiling] Add APIs to clear frame and executable caches (#145499)](#145499) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Joseph Crail","email":"[email protected]"},"sourceCommit":{"committedDate":"2022-11-17T10:04:58Z","message":"[Profiling] Add APIs to clear frame and executable caches (#145499)\n\nThis PR adds two endpoints to clear each respective cache:\r\n\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables`\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes`\r\n\r\nRelated to https://github.com/elastic/prodfiler/issues/2759\r\n\r\n#### Design choices\r\n\r\n1. The `DELETE` method was chosen instead of `PUT` or `POST` since the\r\ngiven semantics of `DELETE` matches the expected behavior for the\r\nrelated issue.\r\n2. Each endpoint will remove all items from the respective cache. A\r\nseparate API for each cache allows us to selectively clear the necessary\r\ncache without the downsides of a catch-all endpoint to clear all caches.\r\nThis gives us the flexibility to add more endpoints if needed. Given the\r\ntradeoff between complexity now or later, it was decided to implement\r\ngeneral invalidation now with the option to invalidate specific items\r\nlater.\r\n3. The RESTful design allows us to clear specific items later (e.g.\r\n`DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}`\r\ncould clear only executable `ID` from the cache).\r\n4. Each endpoint returns an empty payload on success. However, the\r\nKibana logs reflect the actions taken and how many cache items were\r\naffected.\r\n5. The stacktrace cache was ignored since it was not affected by symbols\r\nwritten to Elasticsearch.\r\n6. Each endpoint is not directly accessible from UI since it is expected\r\nthat the endpoints will be called outside of Kibana. However, the\r\nendpoints can be called manually from the browser's console.","sha":"642ab74cfc1a4ef5c0a2dfdadf00bd3792f780a7","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v8.6.0","v8.7.0","v8.5.2"],"number":145499,"url":"https://github.com/elastic/kibana/pull/145499","mergeCommit":{"message":"[Profiling] Add APIs to clear frame and executable caches (#145499)\n\nThis PR adds two endpoints to clear each respective cache:\r\n\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables`\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes`\r\n\r\nRelated to https://github.com/elastic/prodfiler/issues/2759\r\n\r\n#### Design choices\r\n\r\n1. The `DELETE` method was chosen instead of `PUT` or `POST` since the\r\ngiven semantics of `DELETE` matches the expected behavior for the\r\nrelated issue.\r\n2. Each endpoint will remove all items from the respective cache. A\r\nseparate API for each cache allows us to selectively clear the necessary\r\ncache without the downsides of a catch-all endpoint to clear all caches.\r\nThis gives us the flexibility to add more endpoints if needed. Given the\r\ntradeoff between complexity now or later, it was decided to implement\r\ngeneral invalidation now with the option to invalidate specific items\r\nlater.\r\n3. The RESTful design allows us to clear specific items later (e.g.\r\n`DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}`\r\ncould clear only executable `ID` from the cache).\r\n4. Each endpoint returns an empty payload on success. However, the\r\nKibana logs reflect the actions taken and how many cache items were\r\naffected.\r\n5. The stacktrace cache was ignored since it was not affected by symbols\r\nwritten to Elasticsearch.\r\n6. Each endpoint is not directly accessible from UI since it is expected\r\nthat the endpoints will be called outside of Kibana. However, the\r\nendpoints can be called manually from the browser's console.","sha":"642ab74cfc1a4ef5c0a2dfdadf00bd3792f780a7"}},"sourceBranch":"main","suggestedTargetBranches":["8.6","8.5"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/145499","number":145499,"mergeCommit":{"message":"[Profiling] Add APIs to clear frame and executable caches (#145499)\n\nThis PR adds two endpoints to clear each respective cache:\r\n\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables`\r\n* `DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframes`\r\n\r\nRelated to https://github.com/elastic/prodfiler/issues/2759\r\n\r\n#### Design choices\r\n\r\n1. The `DELETE` method was chosen instead of `PUT` or `POST` since the\r\ngiven semantics of `DELETE` matches the expected behavior for the\r\nrelated issue.\r\n2. Each endpoint will remove all items from the respective cache. A\r\nseparate API for each cache allows us to selectively clear the necessary\r\ncache without the downsides of a catch-all endpoint to clear all caches.\r\nThis gives us the flexibility to add more endpoints if needed. Given the\r\ntradeoff between complexity now or later, it was decided to implement\r\ngeneral invalidation now with the option to invalidate specific items\r\nlater.\r\n3. The RESTful design allows us to clear specific items later (e.g.\r\n`DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}`\r\ncould clear only executable `ID` from the cache).\r\n4. Each endpoint returns an empty payload on success. However, the\r\nKibana logs reflect the actions taken and how many cache items were\r\naffected.\r\n5. The stacktrace cache was ignored since it was not affected by symbols\r\nwritten to Elasticsearch.\r\n6. Each endpoint is not directly accessible from UI since it is expected\r\nthat the endpoints will be called outside of Kibana. However, the\r\nendpoints can be called manually from the browser's console.","sha":"642ab74cfc1a4ef5c0a2dfdadf00bd3792f780a7"}},{"branch":"8.5","label":"v8.5.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Joseph Crail <[email protected]> Co-authored-by: Tim Rühsen <[email protected]>
kibanamachine
removed
the
backport missing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds two endpoints to clear each respective cache:
DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executablesDELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/stackframesRelated to https://github.com/elastic/prodfiler/issues/2759
Design choices
DELETEmethod was chosen instead ofPUTorPOSTsince the given semantics ofDELETEmatches the expected behavior for the related issue.DELETE {BASE_KIBANA_PATH}/api/profiling/v1/cache/executables/{ID}could clear only executableIDfrom the cache).