Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove MemoryStorage from Raft Node #2417

Closed
manishrjain opened this issue May 28, 2018 · 2 comments · Fixed by #2433
Closed

Remove MemoryStorage from Raft Node #2417

manishrjain opened this issue May 28, 2018 · 2 comments · Fixed by #2433
Assignees
@manishrjain
Labels
kind/maintenance Maintenance tasks, such as refactoring, with no impact in features.

Comments

@manishrjain
Copy link
Contributor

Badger can power the Storage interface, we should swap out the MemoryStorage, which is used in addition to Badger WAL store. This would decrease the RAM consumption of Dgraph.
@manishrjain manishrjain added the kind/maintenance Maintenance tasks, such as refactoring, with no impact in features. label May 28, 2018
@manishrjain manishrjain self-assigned this May 28, 2018
@jimanvlad
Copy link

jimanvlad commented May 29, 2018 via email

@manishrjain
Copy link
Contributor Author

manishrjain commented May 30, 2018 •
edited
Loading

Hard to say, but should be significant. I know the memory used by the MemoryStorage used to be a problem, so we have ways to avoid it growing too much. Those problems would go away if we don't use it.

This was referenced Jun 1, 2018
Closed
Merged
manishrjain added a commit that referenced this issue Jun 8, 2018
@manishrjain
Badger based Raft Storage (#2433)
872f684 
Instead of using the default RAM based Raft Storage (MemoryStorage), build one based on Badger. This should significantly decrease the RAM usage by the Raft write-ahead log, potentially solving the replication OOM issue at #2424. This also fixes #2417 .

* Clean up the WAL API usage.
* Implemented the Storage interface.
* All raft storage tests work.
* Merge multiple txns into one.
* Optimize FirstIndex by using cache.
* Remove ApplySnapshot, because we no longer need it.
* Correct the order in which we store the Raft entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@manishrjain manishrjain

Labels
kind/maintenance Maintenance tasks, such as refactoring, with no impact in features.
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Badger based Raft Storage hypermodeinc/dgraph
2 participants
@manishrjain @jimanvlad