Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Rework internals #54

Merged
merged 12 commits into from
Dec 21, 2024
Merged

refactor: Rework internals #54

merged 12 commits into from
Dec 21, 2024

Conversation

Ehesp
Copy link
Collaborator

@Ehesp Ehesp commented Dec 18, 2024

Purpose

As discussed on Discord, this is a pretty large change to allow full configurability of Starbase and overhauls some of the internals.

Key changes:

  • The internal DO is now passed around via DataSource.rpc as a single RPC Session, this means for the duration of a single request, we're only charged for a single DO request, whereas currently it's multiple.
  • Internals now have no reference to the environment or request, this is now provided as configuration to the handler (now called StarbaseDB.
  • The handler now accepts a features object (to be documented) where you can disable/enable certain features.
  • The handler now accepts a reworked DataSource instance, with:
    • A rpc property. This is the result of calling the DO init() method to return an RPC session.
    • A source, either external or internal. For worker deployed users, this is still set via the X-Starbase-Source header.
    • A cache boolean property - works is set to true and source is external. For worker deployed users, this is still set via the X-Starbase-Cache header is true.
    • A context property, which is what the RLS rules use. For worker deployed users, decodes the JWT and puts in the payload data. But for custom implementation, they can get the context from wherever they want.
    • An external property, which accepts a ExternalDatabaseSource type, which has all of our providers nicely typed.
  • The expireCache logic has been moved into the internal handler.
  • The handler now accepts a role, of either admin or client. Admin role does not apply allow list or RLS rules. For the deployed worker, this is set by observing the env api keys which are set.

Some additional notes / details:

  • I have removed the MongoDB connection for now, since it doesn't support allow list or RLS due to the syntax differences. Let me know thoughts on this.
  • I'm not sure if we can simply do this: new_sqlite_classes = ["StarbaseDBDurableObject"]? Do we have to tag the migration somehow?
  • I noticed the current implementation checks the source on the URL when connecting via websocket... Shouldn't this be per query on the message level?

I've roughly tested things work.. but with no testing I'm conscious this is a big PR with lots of room to go wrong 👀

@Brayden Brayden added the enhancement New feature or request label Dec 18, 2024
Ehesp and others added 7 commits December 19, 2024 13:06
Changing to "postgresql" as the official Postgres type as the parser library currently requires that format and there are no other dependencies on the alternative.
Update how values are passed into `executeTransaction` so LiteREST calls succeed
Added CORS check back in and updated JWKS logic to support auth requests with JWT
Fix up export and import functionality for internal sources
Needs additional testing before official support
Copy link
Member

@Brayden Brayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a few follow-up commits after testing all of the features. Overall this PR is a quality of life improvements introducing these high-level items, but many more (see original description) included as well.

  • Type safety
  • Ability to toggle on/off features
  • Shared RPC reference for reduced costs

@Brayden Brayden marked this pull request as ready for review December 21, 2024 16:09
@Brayden Brayden merged commit 99567b9 into outerbase:main Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants