Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: speedup and improve cachability of docker build of builder-sd #3430

Merged
merged 13 commits into from
Sep 10, 2024
Merged

fix: speedup and improve cachability of docker build of builder-sd #3430

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d156124
untangle pkg/grpc and core/schema in Transcribe
dave-gray101 Aug 28, 2024
a69bf03
cleanup
dave-gray101 Aug 28, 2024
0ad7782
Merge branch 'master' into fix-pb-transcriptresult
dave-gray101 Aug 29, 2024
0c67ded
stash
dave-gray101 Aug 29, 2024
43f83ea
merge
dave-gray101 Aug 29, 2024
beb135e
drop core/schema
dave-gray101 Aug 29, 2024
2e6c6a1
Merge branch 'master' into fix-speedup-sdbackend
dave-gray101 Aug 29, 2024
4425fcd
fixes
dave-gray101 Aug 29, 2024
1247f45
Merge branch 'master' into fix-speedup-sdbackend
dave-gray101 Aug 30, 2024
3ab2baf
manual merge
dave-gray101 Sep 2, 2024
fe72a21
manual merge
dave-gray101 Sep 2, 2024
071ae57
cleanup
dave-gray101 Sep 2, 2024
3a878e4
re-add `make prepare`
dave-gray101 Sep 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 19 additions & 8 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,14 +263,20 @@ EOT
# In most cases, builder is the image you should be using - however, this can save build time if one just needs to copy backend-assets/grpc/stablediffusion and nothing else.
FROM builder-base AS builder-sd

COPY . .
COPY .git .

RUN make prepare
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmmm the diff here seems misleading, are we skipping now make prepare? Where is this running now?

Copy link
Collaborator Author

@dave-gray101 dave-gray101 Sep 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless I'm wrong - prepare is a dependency when calling make build, so for the actual builder image it just gets ran whenever make decides to

In builder-sd we are explicitly not calling make prepare, as that step takes time to execute - instead, we "fake" it lower down with these first two lines before the build:

RUN make sources/go-stable-diffusion
RUN touch prepare-sources

# Actually build the backend
RUN GRPC_BACKENDS=backend-assets/grpc/stablediffusion make backend-assets/grpc/stablediffusion

As far as my testing showed, the only real reason to have make prepare as an explicit step was to get the sources in place before the stablediffusion/old abseil build was executed

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do call make prepare also to make sure the container has all the sources,so if started with REBUILD it doesn't have to repull sources again. I'm not sure,but I suspect these changes are breaking that behavior (didn't tested)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mudler thanks for explaining this. I've added RUN make prepare to L297 so that it's invoked as a part of the builder stage



# stablediffusion does not tolerate a newer version of abseil, build it first
RUN GRPC_BACKENDS=backend-assets/grpc/stablediffusion make build
# stablediffusion does not tolerate a newer version of abseil, copy only over enough elements to build it
COPY Makefile .
COPY go.mod .
COPY go.sum .
COPY backend/backend.proto ./backend/backend.proto
COPY backend/go/image/stablediffusion ./backend/go/image/stablediffusion
COPY pkg/grpc ./pkg/grpc
COPY pkg/stablediffusion ./pkg/stablediffusion
RUN git init
RUN make sources/go-stable-diffusion
RUN touch prepare-sources

# Actually build the backend
RUN GRPC_BACKENDS=backend-assets/grpc/stablediffusion make backend-assets/grpc/stablediffusion

###################################
###################################
Expand All @@ -285,6 +291,11 @@ COPY --from=grpc /opt/grpc /usr/local
# Rebuild with defaults backends
WORKDIR /build

COPY . .
COPY .git .

RUN make prepare

## Build the binary
## If it's CUDA, we want to skip some of the llama-compat backends to save space
## We only leave the most CPU-optimized variant and the fallback for the cublas build
Expand Down
10 changes: 5 additions & 5 deletions core/backend/transcript.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"github.com/mudler/LocalAI/core/schema"

"github.com/mudler/LocalAI/pkg/grpc/proto"
model "github.com/mudler/LocalAI/pkg/model"
"github.com/mudler/LocalAI/pkg/model"
)

func ModelTranscription(audio, language string, translate bool, ml *model.ModelLoader, backendConfig config.BackendConfig, appConfig *config.ApplicationConfig) (*schema.TranscriptionResult, error) {
Expand All @@ -22,16 +22,16 @@ func ModelTranscription(audio, language string, translate bool, ml *model.ModelL
model.WithAssetDir(appConfig.AssetsDestination),
})

whisperModel, err := ml.BackendLoader(opts...)
transcriptionModel, err := ml.BackendLoader(opts...)
if err != nil {
return nil, err
}

if whisperModel == nil {
return nil, fmt.Errorf("could not load whisper model")
if transcriptionModel == nil {
return nil, fmt.Errorf("could not load transcription model")
}

r, err := whisperModel.AudioTranscription(context.Background(), &proto.TranscriptRequest{
r, err := transcriptionModel.AudioTranscription(context.Background(), &proto.TranscriptRequest{
Dst: audio,
Language: language,
Translate: translate,
Expand Down