Consider to lift off root project requirement

[plastiv]

Thank you for your effort. I would like to initiate a discussion and share a snippet I find useful.

By default ben-manes plugin sets a listener to copy every dependency at every configuration at every submodule with + version. Due to the limitation of Gradle resolving configuration can not be done in parallel. In a typical android kotlin project, google and jetbrains plugins set dozens of configurations per module. Multiplied by 10th of modules this becomes extremely slow to execute. And also pointless waste of resources as the same dependency would be resolved hundreds of times over and over again.

For example, on a project I'm working on with 350 dependencies declared at libs.versions.toml running dependencyUpdates from root takes over 50 minutes to execute.

As an optimization technic, I've found that creating a single configuration and looping through every dependency gives the same new-updates result in under a minute.

Creating an artificial module, called dependency-updates-report with the next content:

// dependency-updates-report/build.gradle
plugins {
    id 'com.github.ben-manes.versions'
}

configurations {
    dependencyUpdateConfiguration
}

dependencies {
    def versionCatalog = project.extensions.getByType(VersionCatalogsExtension).named("libs")
    versionCatalog.dependencyAliases.each {
        versionCatalog.findDependency(it).ifPresent {
            dependencyUpdateConfiguration(it)
        }
    }
}

tasks.named("dependencyUpdates").configure {
    resolutionStrategy {
        componentSelection { rules ->
            rules.all { ComponentSelection selection ->
                boolean rejected = ['alpha', 'beta', 'rc', 'm', 'eap'].any { qualifier ->
                    selection.candidate.version.toLowerCase().contains(qualifier)
                }
                if (rejected) {
                    selection.reject('Alpha, beta, release candidate, milestone and early access preview versions are rejected from report')
                }
            }
        }
    }
}

And running the same ./gradlew dependencyUpdates task will now return updates report times faster.

I believe you would lose the "find unused dependencies" functionality with this approach. This is why I want to initiate discussion, as I'm not sure how useful you consider that to be. I think that tools like https://github.com/autonomousapps/dependency-analysis-android-gradle-plugin could be more effective at findings as it actually checks usage in code, not just declaration at build.gradle files.

Thanks again this plugin has been published at the perfect timing for me as I'm investigating switch to versions catalog.

[plastiv]

Unrelated, but another reason to not apply the plugin to root project is that ben-manes plugin is not configuration cache compatible ben-manes/gradle-versions-plugin#410 and you don't want it to be applied on each invocation to hurt daily usage patterns.

Citation is needed, but I think it will stay incompatible for the foreseen future, as Gradle itself doesn't provide configuration cache compatible API to work with configurations as of yet.

[hvisser]

That's great feedback, exactly what I'm looking for so thanks for that! I realise that the com.github.ben-manes.versions plugin has some limitations, mostly by the fact that it needs to rely on Gradle APIs so that's why I also decided not to try to replicate that functionality and all of the potential issues that come with that.

Do you know if applying the plugin on every submodule (so version report per module) would eliminate some of the performance issues? One approach could be to aggregate all of these reports from the versionCatalogUpdate task (though honestly the resulting json files will probably be huge too so that would probably need some work 😅 )

Alternatively, a dummy module just for updating the catalog could also work, and I think it would in that case if the plugin would just search for all reports across modules.

Re: the unused dependency option, personally I'd like to not have stuff in the catalog that is unused, but I can imagine that not everybody cares or even wants that option so it's not crucial that it's there in all cases I suppose, like the case you describe with a huge amount of modules, so that could be a sacrifice to make in certain situations.

Integrating with the dependency analysis android gradle plugin is also an option, though I'd like the plugin also to work for non-Android projects it seems to support non-Android projects also. Supporting multiple dependency analysis plugins to build upon is probably more work than just supporting one 😅

[plastiv]

Do you know if applying the plugin on every submodule (so version report per module) would eliminate some of the performance issues?

Do you mean as a short term solution? I think ben-manes plugin has implemented scanning of every dependency of every configuration of every submodule because back in the days that was the only way to collect used dependencies at the project with 10th of different ways to declare them with Gradle. With Version Catalog, we don't need to do that it contains the list of dependencies already.

[hvisser]

I'm not sure if I agree with that, because that would assume that all of your (used) dependencies are in the catalog, while you might still accidentally reference dependencies and plugins in your build files by other means, and there's no way to know if the dependencies in the catalog are actually referenced.

But you're right that if you are OK with the above and assume that the catalog itself should just be checked for new versions (regardless of actual usage) then there might be a more efficient way, like your snippet, to collect the dependencies. Maybe I can facilitate that in the plugin somehow. I think there are a few options for that:

• Let the plugin discover / aggregate the report files (that would enable your custom module workaround for example)
• Reconfigure the version updates task somehow so that something similar as in your snippet is automated by the plugin
• Implement a different way to resolve versions (delegating to other plugins or implement something in the plugin)

WDYT?

[plastiv]

Sounds good to me, thank you for the summary!

Since you've just published the plugin let's wait a bit to give people a chance to toy around maybe we will see more feedback on usage patterns before jumping on implementation.

[hvisser]

I did some quick testing on some larger projects and I found something that might be interesting: if the version updates plugin is applied to all subprojects and you then run ./gradlew dependencyUpdates, it's much faster than without applying to the plugin to submodules. I think this is because the subprojects will resolve in parallel now while the root task waits for that to complete (probably due to some kind of locking in place).

If you combine that with the plugin, then you'd need to run ./gradlew dependencyUpdates versionCatalogUpdate to trigger the parallel resolving.

Not saying that this would be the solution, but I am kind of curious if that also brings down the dependency lookup time in your particular case and if that might also be a feasible solution to speed up dependency updates.

[plastiv]

I have made an experiment and yes, an applied ben-manes plugin to every submodule runs fast enough 👍 TIL

[hvisser]

Thanks for testing that! I think that for now that means there's no action required to adjust the plugin, so I'll close the issue. Feel free to reopen if you think that makes sense.