Introduce CLI plugins

[iocanel]

The pull request introduces a plugin mechanism for the Quarkus CLI. This allows to dynamically add commands / subcommand to the CLI.

What is a Plugin

A plugin can be any executable, jar or java command that can be found locally or obtained remotely.
So plugins are classified in the following types:

• Plugins executed via shell
  • binary (any executable prefixed with quarkus- found locally)
• Plugins executed via jbang
  • jar (any runnable jar found locally)
  • jbang alias (any jbang alias prefixed with quarkus- installed locally or through the quarkusio catalog)
  • maven (any maven coordinate in GACTV form pointing to a runnable jar)

How to obtain plugins

Plugins can be found via multiple sources that are described below.

Extension(s)

A quarkus extension may define list of cli-plugins as part of its metadata. The list may contains GACTV string pointing to executable jars.

Limitations: At the moment the cli is able to obtain the list of available extensions, without being very accurate on the exact version of the extension (it uses the version found in the extension catalog).

Local path scanning

Scan the path item for executable files prefixed with quarkus.

Using JBang aliases

Scan the local or project jbang catalog for aliases prefixed with quarkus-.

Using the JBang quarkusio catalog

Scan the quarkusio catalog for aliases prefixed wtih quarkus-.
Note: uses the jbang binary. If missing it will be automatically installed unser .jbang.

Explicitly using the plugin commands

See quarkus plugin add below

Managing plugins

Plugins are managed using the following commands:

quarkus plugin list

The command lists the installed add plugins.

❯ quarkus plug list
No plugins installed!
To include the installable plugins in the list, append --installable to the command.

To list available / installable plugins:

❯ quarkus plug list --installable
    Name              	 Type   	 Location                           	 Description                 	
    greeter          	 binary 	 /home/iocanel/bin/quarkus-greeter 	                             	   	                             	
    fmt               	 jbang  	 quarkus-fmt@quarkusio              	                             	
    kill              	 jbang  	 quarkus-kill@quarkusio          

   Use the 'plugin add' sub command and pass the location of any plugin listed above, or any remote location in the form of URL / GACTV pointing to a remote plugin.	                             		

The output of the list command may be filtered by type using -t or by name using -s flag and a search pattern.
Example: To list all installable plugins that start with the letter k:

❯ quarkus plug list --installable -s "k*"
    Name              	 Type  	 Location                  	 Description                 	
    kill              	 jbang 	 quarkus-kill@quarkusio    	                             		

Use the 'plugin add' sub command and pass the location of any plugin listed above, or any remote location in the form of URL / GACTV pointing to a remote plugin.

quarkus plugin add

To add any of the installable plugins, use quarkus plug add <name or location> :

❯ quarkus plug add kill
Added plugin:
    Name 	 Type  	 Location               	 Description 	
 *  kill 	 jbang 	 quarkus-kill@quarkusio 	             	

The command above installed a plugin by name using the name as listed by quarkus plug list --installable.

The command can be now executed using quarkus kill.

Note: Users are not limited to the plugins discovered by quarkus plug list --installable. Users may install plugins as long as the provide the URL or the Maven coordinates pointing to an executable jar or java file.

Example: Installing an executalbe jar as a plugin via maven coordinates
For this example will use: io.quarkiverse.authzed:quarkus-authzed-cli:runner:jar:0.2.0 which is a a real executable jar that provide a cli utility for the quarkus-authzed extension.

❯ quarkus plug add io.quarkiverse.authzed:quarkus-authzed-cli:runner:jar:0.2.0 -d "Authzed CLI"
Added plugin:
    Name    	 Type  	 Location                                                    	 Description 	
 *  authzed 	 maven 	 io.quarkiverse.authzed:quarkus-authzed-cli:runner:jar:0.2.0 	 Authzed CLI 	       	

Note: It's also possible to set a description that will appear to the help messages.

❯ quarkus --help

Usage: quarkus [-ehv] [--verbose] [-D=<String=String>]... [COMMAND]
...
Commands:
...
  plugin, plug            Configure plugins of the Quarkus CLI.
    list, ls              List CLI plugins.
    add                   Add plugin(s) to the Quarkus CLI.
    remove                Remove plugin(s) to the Quarkus CLI.
    sync                  Sync (discover / purge) CLI Plugins.
  completion              bash/zsh completion:  source <(quarkus completion)
  authzed                 Authzed CLI
...

Where are the plugins added?

Plugins are added in the plugin catalog that lives at: <user home>/.quakrus/cli/plugins/quarkus-cli-catalog.json.

There is a second plugin catalog that is relative to the current project (if available): <project root>/.quarkus/cli/plugins/quarkus-cli-catalog.json.

The effective catalog is the combination of both the user and project catalogs with the latter being able to override entries of the former (e.g. use a different version or location for a plugin).

When managing plugins changes are made to the project catalog unless explicitly specified with the use of --user flag.

quarkus plugin remove

Plugins are removed using quarkus plug remove <plugin name>.

❯ quarkusj plug remove kill
Removed plugin:
    Name 	 Type  	 Location               	 Description 	
    kill 	 jbang 	 quarkus-kill@quarkusio 	             	

quarkus plugin sync

To remove stale plugins or discover new plugins provided by extensions the quarkus plugin sync is available.
With this command binaries and jbang aliases that are added to the catalog but are not longer avaialble will be purged.
Note: Remote plugins that are explicitly added by the user using URL / Maven coordinates are excluded.

The command is also executed implicitly through any of the CLI commands:

• Weekly
• If the project files have been updated since the last catalog update (limited to the module).

Updates

• Introduce a PluginService
• Project catalog now takes precendence.
• Included the scope of the plugin in the output (user/project).
• Sync produces a more detailed output (what's added & what's removed).
• PluginManagement moved to devtools-common theoritically allowed user cli to reuse/rebrand the plugin system.
• Prompt user for automatic plugin installation if requested command is missing but a matching pluign is available
• Prompt user for jbang installtion
• Fix issue with dublicate commands.
• Only use jbang when absolutely necesary
• Support multiple jbang catalogs at once.
• Use quarkus registries to read jbang catalogs from.

[maxandersen]

trying this on - first thing I do running quarkus plugin ls --installable in a code.quarkus project using 2.15.3 quarkus results in:

java.lang.NullPointerException: Cannot invoke "java.util.Optional.orElse(Object)" because the return value of "io.quarkus.cli.plugin.Plugin.getDescription()" is null
	at io.quarkus.cli.plugin.PluginListItem.getDescription(PluginListItem.java:30)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
	at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:662)
	at java.base/java.util.stream.ReferencePipeline.max(ReferencePipeline.java:698)
	at io.quarkus.cli.plugin.PluginListTable.getFormat(PluginListTable.java:110)
	at io.quarkus.cli.plugin.PluginListTable.getContent(PluginListTable.java:63)
	at io.quarkus.cli.plugin.PluginListTable.getContent(PluginListTable.java:33)
	at io.quarkus.cli.plugin.CliPluginsList.listPluigns(CliPluginsList.java:93)
	at io.quarkus.cli.plugin.CliPluginsList.call(CliPluginsList.java:48)
	at io.quarkus.cli.plugin.CliPluginsList.call(CliPluginsList.java:18)
	at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
	at picocli.CommandLine.access$1500(CommandLine.java:148)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
	at picocli.CommandLine.execute(CommandLine.java:2170)
	at io.quarkus.cli.QuarkusCli.run(QuarkusCli.java:76)
	at io.quarkus.cli.QuarkusCli_ClientProxy.run(Unknown Source)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
	at io.quarkus.cli.Main.main(Main.java:9)

[maxandersen]

on plugin list I get:

qss plugin list --installable
    Name                   	 Type   	 Location                                	 Description
    aggregate-pom-gen      	 jbang  	 aggregate-pom-gen
    sqlsh                  	 jbang  	 sqlsh
    quarkus-kill           	 binary 	 /Users/manderse/.jbang/bin/quarkus-kill
    quarkus-fmt@quarkusio  	 jbang  	 quarkus-fmt@quarkusio
    quarkus-kill@quarkusio 	 jbang  	 quarkus-kill@quarkusio

should I not get something like:

qss plugin list --installable
    Name                   	 Type   	 Location                                	 Description
    kill           	 binary 	 /Users/manderse/.jbang/bin/quarkus-kill
    fmt  	 jbang  	 quarkus-fmt@quarkusio
    kill 	 jbang  	 quarkus-kill@quarkusio

aggregate-pom-gen and sqlsh do not start with quarkus - not sure why they show as they are just 2 out of probably 50 other jbang aliases I have.

and then quarkus-kill and quarkus-fmt gets their quarkus- removed since when installed their names are the shorthand?

[maxandersen]

qss plug add whatever gives no feedback and installs a plug without validating.

I would expect qss plug add whatever to fail with "whatever plugin not found, use qss plug list --installable to see what is available" or similar.

[maxandersen]

after doing qss plugin add whatever with nonexistent alias/name I kinda expected qss plugin sync to remove them or at least warn/complain about missing plugins.

[iocanel]

I think that qss plugin add whatever should not work in the first place.

qss plugin sync is meant to remove missing plugins that fall into the following categories:

• jbang alias
• binaries

qss plugin add whatever seems to add it as a jbang alias.
So, yes we have two distinct issues here:

• plugin should not be added
• sync should remove it.

[maxandersen]

we have so many things called catalog so would prefer other name or prefixed somehow. i.e. quarkus-cli-catalog.json

[maxandersen]

should the default action not be that you install in project if inside a project and user/global if outside?

[maxandersen]

should not be printing stacktraces nor catching Exception that broadly?

[iocanel]

Stack traces should definitely not be printed.
IMHO catching Exception in the particular case, does not seem to be an issue.

[maxandersen]

should this not be printed even in case of non dryrun?

[iocanel]

Agree, we need more feeback.

[iocanel]

@maxandersen: Thanks for talking the time to provide such detailed feedback.

I updated the PR and added the following:

• feedback for (add, remove commands)
• only quarkus- prefixed jbang aliases are listed as installable
• sync removes missing jbang aliases
• NPE listing plugins with no description

I will now start looking at your suggestion on shorthandign plugin names and add some more tests.

[maxandersen]

Great. Could you add in the pr description on a high level how the search supposed to work?

[iocanel]

Great. Could you add in the pr description on a high level how the search supposed to work?

Wasn't sure what you meant by how search supposed to work and documented pretty much everything.
Tommorow, I'll move the description to asciidoc. Happy reading.

[github-actions]

🙈 The PR is closed and the preview is expired.

[gastaldi]

Do we need 17 commits for this feature or can they be squashed?

[gsmet]

FWIW, either I have a clean build tomorrow morning and it's included in Beta1 or it will have to wait for CR1 next week.

So squashing is OK if it makes sense as it won't affect the tests but I would refrain from any further changes and implement them in follow-up PRs if they are not critical.

[iocanel]

Do we need 17 commits for this feature or can they be squashed?
Since Feb 15 where I initially created this pull request I've been continously adding features. I've been also regularly squashing my fixup and refactoring commits. The result is these 17 commits. I don't mind further grouping commits and squashing them to lower that number.

[quarkus-bot]

Failing Jobs - Building 5978644

Status	Name	Step	Failures	Logs	Raw logs
⌛	JVM Tests - JDK 11	Build	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 17	Build	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 17 Windows	Build	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 19	Build	⚠️ Check →	Logs	Raw logs
✖	MicroProfile TCKs Tests	Verify	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ MicroProfile TCKs Tests #

- Failing: tcks/microprofile-lra 

📦 tcks/microprofile-lra

✖ org.eclipse.microprofile.lra.tck.TckParticipantTests.testNonJaxRsCompletionStageVoid line 145 - More details - Source on GitHub

java.lang.AssertionError: Non JAX-RS @Compensate method with CompletionStage<Void> should have been called expected:<1> but was:<0>
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)

[iocanel]

@gsmet @maxandersen: errors seem to be irrelevant. Should we merge?

[maxandersen]

+1, error is a jvm dump in microprofile which doesn't touch this code afaics.