containerizingMode = "packaged" fails when jar is not already present

[mar-kolya]

Environment:

• Jib version: 2.5.0
• Build tool: gradle
• OS: macos/linux

Description of the issue:

When containerizingMode = "packaged" is enabled jib tasks fail with Obtaining project build output files failed.

Expected behavior:

Build should work. :)

Steps to reproduce:

1. Set containerizingMode = "packaged"
2. build container with gradle
3. Observe failure

Additional Information:

Jib tasks should depend on jar task if containerizingMode = "packaged" is used otherwise there is no jar unless some other task builds it. Adding dependsOn('jar') works around the issue.

[loosebazooka]

That's strange, we already do depend on the 'jar' task. Do you have a complicated build? Can you share your build.gradle with us?
In the failure config can you run ./gradlew jib --dry-run and share that output as well?

[mar-kolya]

Unfortunately I do not have a reproducer available that I can share at the moment. We do have somewhat complex setup with multiple subproject and 'centralized' config in 'allprojects' section on the top level.

As far as I can see jib depends on classes, but not on jar - even with containerizingMode = "packaged"

[loosebazooka]

I cannot reproduce this locally. Jib with containerizingMode="packaged" automatically adds the jar task as a dependency in my tests.

You will have to provide us with a breaking example to move forward. I only suspect that it has something to do with your build config. (perhaps jar is disabled?) Did you try ./gradlew jib --dry-run ?

[mar-kolya]

So default java-gradle project from idea with this gradle config:

plugins {
    id 'java'
    id 'com.google.cloud.tools.jib' version '2.5.0'
}

group 'org.example'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.12'
}

afterEvaluate {
    jib {
        from {
            image = "base-image"
        }
        containerizingMode = "packaged"
        container {
            appRoot = "/usr/local/app"
        }
    }
}

reproduces the issue.

Dry-run output:

$./gradlew jib --dry-run
:compileJava SKIPPED
:processResources SKIPPED
:classes SKIPPED
:jib SKIPPED

Things work fine without afterEvaluate, but in our project we need it because we need some other plugins applied first.

[loosebazooka]

It should not be required to use the afterEvaluate block. What version of gradle are you using?

[mar-kolya]

We use 6.6.1

afterEvaluate is not needed in above example. It is however needed in specific project we are using jib for: we have subprojects and 'common' configuration for them in the root project - where we define jib settings. Due to some setup complications this requires afterEvaluate.

[loosebazooka]

I suspect using afterEvaluate is triggering something too late in the JibPlugin config. Because we are configuring our own task in an afterEvaluate block. If our code in our afterEvaluate runs first, then we cannot parse out the value of containerizingMode because it's not populated until your afterEvaluate block is run.

What exactly are you configuring in Jib that requires it to be evaluated later. We have some work in progress to delay evaluation of certain jib parameters: #2727 , so you can use providers to pass in parameters at a later time?

[mar-kolya]

So we have quite complicated setup, but one thing that we have is as following:

• We have custom config for a dependency (i.e. not runtime, build or test - custom dependency type).
• We have a special task that puts that dependency into a directory.
• We ask jib to use that directory as extra directory.

Not using afterEvaluate results in dependency being resolved too early and gradle complaining about not being able to configure a sub project.

I'll see if I can move jib config out of afterEvaluate but last time I tried things had fallen apart.

[loosebazooka]

I think you can use this while we figure out lazy evaluation.
You can specify that one item in afterEvaluate and the rest of the jib items outside of it.

jib {
  // stuff like containerizerMode
}
project.afterEvaluate {
  jib {
    // stuff like extraDirectories
  }
}

While I don't think it is ever ideal to configure something in the afterEvaluate block, I think this is mostly safe.

[mar-kolya]

Yeah, splitting jib config into two pieces seems to work. Thanks.

Still non-obvious... :)

[chanseokoh]

@loosebazooka based on your comments (#2732 (comment) and #2732 (comment)), how do you think this particular issue can be resolved best? And in general?

[loosebazooka]

I think containerizingMode is a special case that affects the build tree (rather than just the jib process). For everything else, afterEvaluate can be a fine stopgap while we eventually move everything to a lazy evaluation model?

[chanseokoh]

I see, so basically we do intend to move all the Gradle parameters to support lazy evaluation (as done in #2727 and #2905).

[faireai-fmonorchio]

I have a gradle project configured as is:
keycloak
|--- keycloak-add-on1
|--- keycloak-add-on2

Where, each subprojects build a jar and after it is copied in build/libs of the parent. The parent has jar task disabled (because there are no sources) and use jib with containerMode packaged to copy the jars inside docker image but fails because jar is disabled. There is a solution for this? thanks

[emmileaf]

While containerizingMode cannot be configured inside an afterEvaluate block, lazy evaluation of extraDirectories parameters is now enabled in #3737. Closing this issue given this and the workaround, please re-open if needed.

(@faireai-fmonorchio, please consider opening a new issue with a minimum reproducible example if your question is still relevant, since this issue appears to be a bit different from your described use case. Thanks!)