Skip to content

Commit

Permalink
SDK-as-Modules refactor + Python Support (#5900)
Browse files Browse the repository at this point in the history
* engine: remove old filesync hacks no longer needed

After some upstream changes we can remove a nice chunk of code that was
working around the fact that buildkit was overriding context when using
export functionality.

Signed-off-by: Erik Sipsma <[email protected]>

* engine: support filesync w/ engine container; container asTarball API

This change enables us to interact with engine container's filesystem
the same way we do client filesystems in terms of file import+export.

The motivation was primarily to support some upcoming requirements
around SDK bootstrapping where builtin SDKs will be pre-loaded into the
engine container, which allows module usage w/out a hard dependency on
the internet (at least when using builtin module SDKs).

This also includes a new user-facing API that enables converting a
Container object to a File representation of it as an exported tarball.
This API will be very convenient to have for the upcoming work and be
useful for users in general. It saves you from having to export a
tarball to your local filesystem and then load it back into the engine,
instead doing that whole process internally and more efficiently by not
having to cross the wire all the way to the client and back.

That API is also helpful as some initial validation that the engine
container filesync is working as expected.

Signed-off-by: Erik Sipsma <[email protected]>

* first pass sdk-as-module refactor passing existing tests

Signed-off-by: Erik Sipsma <[email protected]>

* Add prototype for bootstrapping a python container with

This is just the last iteration of an attempt to understand what's needed to bootstrap an sdk runtime.

Added a few vars to error messages in `core/module.go` and `core/schema/module.go` to make it easier to debug.

Signed-off-by: Helder Correia <[email protected]>

* wip python working e2e

Signed-off-by: Erik Sipsma <[email protected]>

* Fix python sdk

Signed-off-by: Helder Correia <[email protected]>

* fix hardcode of root+subpath in core core for builtin modules

Signed-off-by: Erik Sipsma <[email protected]>

* cleanup various path handling

Signed-off-by: Erik Sipsma <[email protected]>

* add initial e2e test for python modules

Signed-off-by: Erik Sipsma <[email protected]>

* share engine container paths rather than hardcode twice

Signed-off-by: Erik Sipsma <[email protected]>

* remove unused module env vars

Signed-off-by: Erik Sipsma <[email protected]>

* add ModuleConfig to graphql api to support SDK Module implementations

Signed-off-by: Erik Sipsma <[email protected]>

* doc more code

Signed-off-by: Erik Sipsma <[email protected]>

* fix go+nodejs generate

Signed-off-by: Erik Sipsma <[email protected]>

* Revert Elixir codegen

Signed-off-by: Thanabodee Charoenpiriyakij <[email protected]>

* Temporary quick fix for Elixir codegen

Signed-off-by: Thanabodee Charoenpiriyakij <[email protected]>

* appease elixir lint

Signed-off-by: Erik Sipsma <[email protected]>

---------

Signed-off-by: Erik Sipsma <[email protected]>
Signed-off-by: Helder Correia <[email protected]>
Signed-off-by: Thanabodee Charoenpiriyakij <[email protected]>
Co-authored-by: Helder Correia <[email protected]>
Co-authored-by: Thanabodee Charoenpiriyakij <[email protected]>
  • Loading branch information
3 people authored Oct 17, 2023
1 parent 17bd90a commit 1dfec91
Show file tree
Hide file tree
Showing 62 changed files with 146 additions and 10,877 deletions.
17 changes: 0 additions & 17 deletions cmd/bootstrap/help.txt

This file was deleted.

243 changes: 0 additions & 243 deletions cmd/bootstrap/main.go

This file was deleted.

95 changes: 0 additions & 95 deletions cmd/codegen/main.go

This file was deleted.

Loading

0 comments on commit 1dfec91

Please sign in to comment.