fix(pluginservers): restart all instances of a plugin #11306
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gszr
force-pushed
the
fix/pluginserver-instance-cleaning
branch
from
934ee62 to
785bf34
Compare
When the plugin -- which acts as a plugin server -- dies for some reason, all entities associated with it must be invalidated, not only the current. If two instances A and B of an external plugin P exist (e.g., if applied to different route objects), they receive a unique ID [1] defined by the the plugin server when the instance is started. If A and B are started sequentially and are the first instances, they get IDs 0 and 1, respectively. This ID is cached on the Kong side by the plugin server manager [2]. If plugin P dies, Kong plugin server manager will detect this condition and restart it. As it was, the code would remove whichever instance first detected the condition and leave the other in `running_instances`; once the plugin server was up, the removed instance would be restarted, in some cases with the same Id already in use by the stale instance that should have been removed from `running_instances`. Subsequent requests may then run the wrong instance of the external plugin. This change fixes the issue by ensuring that both instances A and B are removed from the `running_instances` table, [1] https://github.com/Kong/go-pdk/blob/4deec45a5bfe421a0ceebcbf7931d395250949b0/server/instance.go#L47 [2] `kong/runloop/plugin_servers/process.lua`
gszr
force-pushed
the
fix/pluginserver-instance-cleaning
branch
from
785bf34 to
85db724
Compare
flrgh
reviewed
Jul 27, 2023
flrgh
approved these changes
Jul 27, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NOTE to reviewers: this fix has already been merged into Enterprise.
When the plugin -- which acts as a plugin server -- dies for some reason, all entities associated with it must be invalidated, not only the current.
If two instances A and B of an external plugin P exist (e.g., if applied to different route objects), they receive a unique ID [1] defined by the the plugin server when the instance is started. If A and B are started sequentially and are the first instances, they get IDs 0 and 1, respectively. This ID is cached on the Kong side by the plugin server manager [2]. If plugin P dies, Kong plugin server manager will detect this condition and restart it.
As it was, the code would remove whichever instance first detected the condition and leave the other in
running_instances; once the plugin server was up, the removed instance would be restarted, in some cases with the same Id already in use by the stale instance that should have been removed fromrunning_instances. Subsequent requests may then run the wrong instance of the external plugin. This change fixes the issue by ensuring that both instances A and B are removed from therunning_instancestable,[1] https://github.com/Kong/go-pdk/blob/4deec45a5bfe421a0ceebcbf7931d395250949b0/server/instance.go#L47
[2]
kong/runloop/plugin_servers/process.lua