Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python API to match R package functionalities #11

Closed
grst opened this issue Aug 22, 2024 · 0 comments · Fixed by #30
Closed

Python API to match R package functionalities #11

grst opened this issue Aug 22, 2024 · 0 comments · Fixed by #30

Comments

@grst
Copy link
Collaborator

grst commented Aug 22, 2024

Implement stage_here and read_params in Python

Open questions

Shall this be implemented as a public API in the dso_core package that directly accesses the same functionality as the CLI or as a separate dso-python package that communications with the dso CLI as the R package does?

Option 1: Python API of the dso_core package

See draft PR here: #30

Advantages: Less overhead, everything is already there. Less can go wrong than when calling a binary (e.g. PATH not correctly set etc)

Disadvantage: dso_core needs to be installed in the environment that runs Python. This is fine as long as we have one environment per project (e.g. managed by uv, see #6). If we start having multiple environment for different stages, it becomes tricky:

  • it requires dso to be installed and potentially adds a lot of dependencies that would not be needed otherwise
  • there could be version mismatches between dso in the venv and dso in the project. Like that, when compiling configs from the venv, it could in the worst case generate different results than when compiling from outside.
    • possibly still fine when running dso repro: This will compile the configs before running anything and sets an environment variable that ensures the configs are not compiled again.

Option 2: Separate package that communicates via CLI

This would be a very lightweight package with no dependencies (except pyyaml and uv probably).
When calling the read_params, it would delegate the call to dso get-config, as done by the R package.

Instead of directly calling a dso binary, we could call uv run -- dso instead. This saves an extra hoop through the dso-manager and it's anyway only valid within a project. If the user doesn't want to use uv, it could still call dso directly.

Advantages: Stage environments can be isolated from the project environment and still use a consistent dso version across the entire project.

Disadvantages: Potentially something can go wrong when calling a binary. Less so when using uv that's installed in the env. Also potentially a bit slower on a slow file system.

Option 3: both

Technically, the two approaches are not mutually exclusive. Although there is an additional maintenance burden and complexity, so this isn't anything we'd do initially.

--> for now go with Option 1

@grst grst mentioned this issue Oct 4, 2024
@grst grst closed this as completed in #30 Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant