Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dbless persistence #8670

Merged
merged 3 commits into from
May 25, 2022
Merged

Add dbless persistence #8670

merged 3 commits into from
May 25, 2022

Conversation

aboudreault
Copy link
Contributor

This PR makes the dbless persistent between restarts (lmdb). If a config exists in the db, load it and build the router.

With this addition, we can remove the config cache (config.cache.tar.gz) (I will open another PR for this).

@aboudreault aboudreault requested a review from dndx April 12, 2022 01:33
@aboudreault aboudreault force-pushed the feat/dbless-persistence branch from 18dc7d1 to 155a546 Compare April 12, 2022 01:36
@aboudreault aboudreault force-pushed the feat/dbless-persistence branch from 155a546 to 6b02995 Compare April 13, 2022 14:39
@github-actions github-actions bot removed the core/db label Apr 13, 2022
@aboudreault aboudreault force-pushed the feat/dbless-persistence branch from 6b02995 to c595b35 Compare April 13, 2022 14:51
@github-actions github-actions bot removed the core/db label Apr 13, 2022
@aboudreault aboudreault mentioned this pull request Apr 20, 2022
@aboudreault aboudreault added this to the 3.0 milestone May 5, 2022
@RobSerafini RobSerafini requested review from mayocream and gszr May 6, 2022 20:42
kong/init.lua Outdated Show resolved Hide resolved
@aboudreault aboudreault requested a review from a team as a code owner May 11, 2022 12:20
@dndx
Copy link
Member

dndx commented May 11, 2022

@bungle is it possible for you to take a closer look at this (to make sure we are not overlooking anything)? Especially make sure the reload doesn't mess up anything.

@aboudreault aboudreault added the pr/ready This PR is considered ready and can be merged at anytime (given it received no subsequent changes) label May 12, 2022
assert.res_status(404, res) -- 404, only the declarative config is loaded
end)

it("doesn't load the persisted lmdb config if a declarative config is set on reload", function()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For good measure, I think it would be worthwhile to have a test case that covers worker process replacement (kill -9 an existing worker and then validate expected behavior) as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flrgh would you consider this test a blocker? I thought that it will basically do the same thing than an initial worker start, cause the code is right in init_worker.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah I wouldn't go as far as to say it's a blocker. I don't know this code as well as you do, so I think this is up to your discretion whether more tests are needed before merging. To clarify, there's nothing in particular about this changeset that makes me believe it will introduce any new bugs. I just think our test coverage of single worker replacement is lacking overall (it's a very common blind spot I've observed in OpenResty code), so I'll almost always advocate for adding more tests when init_worker is concerned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense @flrgh! I will check to add one before merging.

Copy link
Contributor Author

@aboudreault aboudreault May 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flrgh since I am working on other tasks and we want to unblock as many PR merges as possible before the feature freeze, I will merge this now, but still consider your test case addition is worth to add.

@gszr gszr requested a review from bungle May 18, 2022 16:37
@aboudreault aboudreault force-pushed the feat/dbless-persistence branch from 058f19a to dc1f746 Compare May 25, 2022 13:46
@aboudreault aboudreault merged commit fe20857 into master May 25, 2022
@aboudreault aboudreault deleted the feat/dbless-persistence branch May 25, 2022 14:04
aboudreault added a commit that referenced this pull request Aug 16, 2022
ms2008 added a commit that referenced this pull request Jan 18, 2023
windmgc pushed a commit that referenced this pull request Mar 1, 2023
AndyZhang0707 added a commit that referenced this pull request Aug 24, 2023
AndyZhang0707 added a commit that referenced this pull request Sep 18, 2023
dndx pushed a commit that referenced this pull request Sep 19, 2023
AndyZhang0707 added a commit that referenced this pull request Dec 4, 2023
windmgc pushed a commit that referenced this pull request Dec 11, 2023
AndyZhang0707 added a commit that referenced this pull request Jul 18, 2024
AndyZhang0707 added a commit that referenced this pull request Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/ready This PR is considered ready and can be merged at anytime (given it received no subsequent changes) size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants