Does electric support partitions/shards?

[iyobo]

One of the benefits of a pouchDB/couchDB data stack is that, for user groups with isolated data (companies, teams of users, etc), partitioning is very natural as each isolated dataset can be put in its own "TableBase" (single-table database).

A system where every user group's data accumulates in a PG table is an eventual issue without eventual partitioning.
Does electric support partitioned tables on postgres? What about sharding? e.g with the tidus extension?

[balegas]

From the standpoint of an electric app, the local database will only contain the subset of data that is being synched via shape subscriptions (e.g. the relevant data for the group). On the client you get the electrified schema as defined in Postgres, but performance is not tied to the backend database size or workload patterns.

It is important to scale the backend database, but at the moment, partitioning or sharding would greatly depend on how transparent extensions are with regards to schema management and logical replication. We haven’t explored more for now.

[iyobo]

I suppose Would partitioning just be a responsibility of the database then?

[balegas]

If extension is fully transparent and supports logical replication then the answer is yes. In practice, I expect this to be ore nuanced and extra work would likely be required.

[KyleAMathews]

👋 we've been working the last month on a rebuild of the Electric server over at a temporary repo https://github.com/electric-sql/electric-next/

You can read more about why we made the decision at https://next.electric-sql.com/about

We're really excited about all the new possibilities the new server brings and we hope you'll check it out soon and give us your feedback.

We're now moving the temporary repo back here. As part of that migration we're closing all the old issues, PRs, and discussions. We really appreciate you taking the time to start this discussion! If it's still relevant in the new system, please reopen.