You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
LazyModel makes it rather complicated for someone to add their own model, especially when some changes need to be made to the input/output. The reason LazyModel exists is so that we can spawn new processes and share a PipelineConfig instance. This is not possible without lazy loading because copying weights between processes is expensive and could also fail.
Idea
To get rid of LazyModel without sacrificing multiprocessing we need to lighten the content of PipelineConfig, which currently needs to hold model objects. We could simply hold model names or paths and share that across processes. Then, each process will have to load the model given the path or name.
This is actually the same behavior that exists today, the difference lies in the fact that lazy loading will be left to the config/pipeline instead of the actual model.
The text was updated successfully, but these errors were encountered:
Problem
LazyModel
makes it rather complicated for someone to add their own model, especially when some changes need to be made to the input/output. The reasonLazyModel
exists is so that we can spawn new processes and share aPipelineConfig
instance. This is not possible without lazy loading because copying weights between processes is expensive and could also fail.Idea
To get rid of
LazyModel
without sacrificing multiprocessing we need to lighten the content ofPipelineConfig
, which currently needs to hold model objects. We could simply hold model names or paths and share that across processes. Then, each process will have to load the model given the path or name.This is actually the same behavior that exists today, the difference lies in the fact that lazy loading will be left to the config/pipeline instead of the actual model.
The text was updated successfully, but these errors were encountered: