You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Shall this be implemented as a public API in the dso_core package that directly accesses the same functionality as the CLI or as a separate dso-python package that communications with the dso CLI as the R package does?
Advantages: Less overhead, everything is already there. Less can go wrong than when calling a binary (e.g. PATH not correctly set etc)
Disadvantage: dso_core needs to be installed in the environment that runs Python. This is fine as long as we have one environment per project (e.g. managed by uv, see #6). If we start having multiple environment for different stages, it becomes tricky:
it requires dso to be installed and potentially adds a lot of dependencies that would not be needed otherwise
there could be version mismatches between dso in the venv and dso in the project. Like that, when compiling configs from the venv, it could in the worst case generate different results than when compiling from outside.
possibly still fine when running dso repro: This will compile the configs before running anything and sets an environment variable that ensures the configs are not compiled again.
Option 2: Separate package that communicates via CLI
This would be a very lightweight package with no dependencies (except pyyaml and uv probably).
When calling the read_params, it would delegate the call to dso get-config, as done by the R package.
Instead of directly calling a dso binary, we could call uv run -- dso instead. This saves an extra hoop through the dso-manager and it's anyway only valid within a project. If the user doesn't want to use uv, it could still call dso directly.
Advantages: Stage environments can be isolated from the project environment and still use a consistent dso version across the entire project.
Disadvantages: Potentially something can go wrong when calling a binary. Less so when using uv that's installed in the env. Also potentially a bit slower on a slow file system.
Option 3: both
Technically, the two approaches are not mutually exclusive. Although there is an additional maintenance burden and complexity, so this isn't anything we'd do initially.
--> for now go with Option 1
The text was updated successfully, but these errors were encountered:
Implement
stage_here
andread_params
in PythonOpen questions
Shall this be implemented as a public API in the dso_core package that directly accesses the same functionality as the CLI or as a separate
dso-python
package that communications with the dso CLI as the R package does?Option 1: Python API of the dso_core package
See draft PR here: #30
Advantages: Less overhead, everything is already there. Less can go wrong than when calling a binary (e.g. PATH not correctly set etc)
Disadvantage: dso_core needs to be installed in the environment that runs Python. This is fine as long as we have one environment per project (e.g. managed by uv, see #6). If we start having multiple environment for different stages, it becomes tricky:
dso repro
: This will compile the configs before running anything and sets an environment variable that ensures the configs are not compiled again.Option 2: Separate package that communicates via CLIThis would be a very lightweight package with no dependencies (except
pyyaml
anduv
probably).When calling the
read_params
, it would delegate the call todso get-config
, as done by the R package.Instead of directly calling a dso binary, we could call
uv run -- dso
instead. This saves an extra hoop through thedso-manager
and it's anyway only valid within a project. If the user doesn't want to useuv
, it could still calldso
directly.Advantages: Stage environments can be isolated from the project environment and still use a consistent dso version across the entire project.
Disadvantages: Potentially something can go wrong when calling a binary. Less so when using
uv
that's installed in the env. Also potentially a bit slower on a slow file system.Option 3: bothTechnically, the two approaches are not mutually exclusive. Although there is an additional maintenance burden and complexity, so this isn't anything we'd do initially.
--> for now go with Option 1
The text was updated successfully, but these errors were encountered: