Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server-side package installation #2683

Merged
merged 17 commits into from
Mar 27, 2024
Merged

Server-side package installation #2683

merged 17 commits into from
Mar 27, 2024

Conversation

joeyballentine
Copy link
Member

This PR implements installing packages through the backend, by having the host server kill the worker server, install the dependencies, then start the worker back up. Since it is now using our pip fork, it can also report download progress directly through pip.

This means the frontend no longer handles any pip dependency installation. The only remnant of this is the getFindLinks function used for manual/copy mode, which i will have the backend handle in a future PR.

The way it reports progress right now is a bit jank. I want to redo this in the future but i also wanted to make minimal changes at the moment so I will handle that once I get around to redoing the entire dependency manager page.

This works well enough for now.

Copy link
Member

@RunDevelopment RunDevelopment left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only briefly looked at this, but did you remove the ability to install/update individual pypi packages? The API only deals with chainner packages.

backend/src/server_host.py Show resolved Hide resolved
backend/src/server_host.py Outdated Show resolved Hide resolved
src/common/Backend.ts Outdated Show resolved Hide resolved
@joeyballentine
Copy link
Member Author

but did you remove the ability to install/update individual pypi packages?

We only ever installed things on a chaiNNer-package-level basis, so to simplify the process I made it go by chaiNNer package.

Think of the install/uninstall process as installing or installing an extension rather than installing and uninstalling just the dependencies. Maybe in the future an extension would be more than just pypi packages.

Do you have a usecase for being able to manage individual deps like that?

@RunDevelopment
Copy link
Member

Do you have a usecase for being able to manage individual deps like that?

PyTorch. Spandrel updates are necessary, pytorch less so. So when we do a pytorch update, I typically stay with the older version purely because I don't want to wait for the download.

@joeyballentine
Copy link
Member Author

But that's never been possible within chaiNNer afaik, how were doing that?

@RunDevelopment
Copy link
Member

I happen to contribute to this project, so I get dep changes a little more frequently.

But you're right. I'm probably the only one with this problem, so let's not complicate things. I can still get the pip command and edit it if need be.

@joeyballentine joeyballentine merged commit bb817e4 into main Mar 27, 2024
17 checks passed
@joeyballentine joeyballentine deleted the server-side-dep-install branch March 27, 2024 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants