Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core refactors #126

Merged
merged 18 commits into from
Jun 22, 2023
Merged

Core refactors #126

merged 18 commits into from
Jun 22, 2023

Conversation

mikeknep
Copy link
Contributor

@mikeknep mikeknep commented Jun 16, 2023

No user-facing changes here, strictly an internal refactor.

Previously, tables with JSON columns got "extra special treatment": after creating invented tables from them, we added the invented tables to the RelationalData graph as nodes, but not the source table itself; instead, metadata about that original table was stored in a RelationalJson object responsible for knowing everything necessary to transform invented tables back to the original shape. Then the RelationalData stored a dictionary of those objects. This was always a bit clunky and added a lot of mental overhead when trying to reason about the internal state.

In the spirit of "cattle, not pets", this PR makes what we now call "producer" tables more similar to normal and invented tables. Probably the key takeaway is that every table is a node in the graph, whether it is a "normal" table, an invented table, or a producer of invented tables. All state for a given table is stored on the graph node. This brings several benefits:

  • the JSON module, with all its (de)normalizing transformation functions, is now stateless—we store all the table-mapping information on the producer table graph node
  • backups no longer need to care about this extra collection of objects (RelationalJsons)—they just write down everything known about all tables, and (by extension) the restore process is simplified. (RelationalJson backup was also a bit "lossy"; no more!)
  • elevating the concept of "producer tables" helps establish a more sustainable pattern for if we ever invent tables for other reasons in the future

There are also a few internal helper methods added to generally make things more readable.

@mikeknep mikeknep merged commit 76984aa into main Jun 22, 2023
@mikeknep mikeknep deleted the core-refactors branch June 22, 2023 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants