Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have a separate layers for SNAPSHOT dependencies #735

Closed
FrogDevelopper opened this issue Jun 5, 2023 · 1 comment · Fixed by #903
Closed

Have a separate layers for SNAPSHOT dependencies #735

FrogDevelopper opened this issue Jun 5, 2023 · 1 comment · Fixed by #903
Assignees

Comments

@FrogDevelopper
Copy link

FrogDevelopper commented Jun 5, 2023

Feature description

As explained in How are Jib applications layered?, we could have a separated layer for the "SNAPSHOT" dependencies from the other so to not update the "static" dependencies layer that does not change between 2 builds

I've been able to do it locally by overloading the BuildLayersTask task

    withType<BuildLayersTask> {
        doLast {
            // moving SNAPSHOT into a different layer
            copy {
                from(outputDir.dir("libs"))
                    .include("*-SNAPSHOT.jar")
                    .into(outputDir.dir("snapshotsLibs"))
            }
            delete {
                delete(fileTree(outputDir.dir("libs")).matching {
                    include("*-SNAPSHOT.jar")
                })
            }
        }
    }

    withType<MicronautDockerfile> {
        copyFile("layers/snapshotsLibs", "/home/app/libs")
}

but I feel that would be a great feature to be included by default

@melix melix self-assigned this Dec 7, 2023
@melix
Copy link
Collaborator

melix commented Dec 7, 2023

I am working on something like this, which would actually make the difference between:

  • external dependencies
  • external dependencies with SNAPSHOT
  • project dependencies (e.g in a multiproject build)
  • application

And making use of COPY --link to make things even faster.

melix added a commit that referenced this issue Dec 8, 2023
This commit improves our docker file generation in order to use
multiple layers for dependencies. Instead of having a single libs
directory, we now have 3 layers which are copied to the `libs`
directory:

- one for the transitive dependencies which are not snapshot
- one for the snapshot dependencies
- one for the project dependencies

In addition to the layer which contains the application jar.
Moreover, the dockerfile creation now makes use of the `--link`
option for COPY, which makes it even more efficient.

Fixes #735
melix added a commit that referenced this issue Dec 11, 2023
* Create multi-layer docker files

This commit improves our docker file generation in order to use
multiple layers for dependencies. Instead of having a single libs
directory, we now have 3 layers which are copied to the `libs`
directory:

- one for the transitive dependencies which are not snapshot
- one for the snapshot dependencies
- one for the project dependencies

In addition to the layer which contains the application jar.
Moreover, the dockerfile creation now makes use of the `--link`
option for COPY, which makes it even more efficient.

Fixes #735

* Use `--link` for all COPY operations

* Fix CRaC docker files

* Disable `--link` by default

As this is causing some flakiness on CI, it could probably cause flakiness
on user builds too.

* Do not use COPY for empty layers

* Restore --link by default

Since it wasn't at fault when running builds on CI.

* Restore lenient(false)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants