feat(ray): support containerized model serving #114
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Because - `model-backend` needs `Ray` CLI to deploy dockerized application This commit - return deployment handle for CLI to reference
Because - we need to provide easy-to-use script for user to build and push containerized model to desired registry This commit - add `docker` dependency - add `build` module script for easy image building and pushing
Because - we need to copy model weight files along with config and model.py This commit - update `dockerfile` to copy all files in the same directory
Because - It is not practical to determine vram usage solely from model file size This commit - expose cpu/gpu/ram resource allocation config to user
Because - It is hard to know what went wrong without build logs - pip install tends to timeout for large packages installation This commit - print build logs - add default timeout for pip package installation
Because - we remove the restriction of the model folder structure after deprecating triton model support This commit - update custom model guide
Because - user may want to push to multiple registries, it is undesirable to define in `instill.yaml` This commit - separate `build` and `push` script - remove `registry` from model config
heiruwu
added a commit
that referenced
this pull request
Mar 20, 2024
Because - We are going to support containerized model serving with `Instill Model` This commit - add deployment handle return - add modules for building and pushing model image - expose cpu/gpu/memory resource allocation configs - update `instill_deployable` decorator
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because
Instill ModelThis commit
instill_deployabledecorator