Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider global migration from Postgres #189

Open
telezhnaya opened this issue Nov 9, 2021 · 0 comments
Open

Consider global migration from Postgres #189

telezhnaya opened this issue Nov 9, 2021 · 0 comments
Assignees

Comments

@telezhnaya
Copy link
Contributor

telezhnaya commented Nov 9, 2021

We've already discussed this topic several times, it's better to collect all thoughts/requirements/ideas in one place so that we can make the decision faster when we decide to move on with the migration.

  1. ### Why do we think about the migration? 1. The speed of SELECT queries degraded dramatically, and we have the feeling that it couldn't be improved in the current setup 2. I'm afraid that in one day we will have issues with the speed of INSERTs (it's not the bottleneck still; if we will meet this problem, the hotfix is to add the partitions)

  2. ### The results of the discussions

  • We anyway need to use relational DB: it suits the nature of the data perfectly
  • Partitioning does not solve the main problem, we still have the same CPU limit
  • We are thinking about the sharded solution (read: partitioning on several machines), the main candidate for now is https://www.singlestore.com
  • It would be great to have the ability to share the dump of the DB to help the users run their own Indexer without the requirement to wait several weeks for collecting all historical data (see Postgres dump #187)
  • It's not the burning issue; it's still possible to solve all our problems with the current solution + https://github.com/near/near-analytics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants