refactor: change plugin system to use gRPC (#3529)

* chore: update plugin template Go version to 1.20 * docs: add documentation to clarify the plugin cache purpose * chore: improve plugins code spacing for better redability * chore: add context support to plugin scaffold * refactor: move plugin RPC implementation to an `rpc.go` file * refactor: simplify RPC plugin implementation * feat: add proto buffer files for the gRPC plugins * chore: generate plugin files from proto * chore: remove extra blank line * chore: add Flags to ExecutedCommand proto definition * refactor: remove rpc plugin implementation * refactor: move interface types methods to the generated types * refactor: move plugin interface into grpc package * feat: add gRPC plugin implementation * refactor: change plugin to use gRPC for communication * refactor: change plugin command to use the new gRPC interface * refactor: add context to gRPC plugin client communication * refactor: move plugin interface and gRPC implementation Done to have a better package API. * chore: add plugin flag type aliases for v1 * refactor: add flag contructor methods to executed command * refactor: update plugin template to use gRPC * test: fixed tests * chore: move proto flag type enum definition into flag message * chore: add changelog entry * chore: correct flag type reference in plugin service tests * test: improve plugin scaffold test * test: change flag types * chore: lower plugin template Go version to 1.19 This is temporary until CI/CD allows Go v1.20 * chore: use `hplugin` alias go hashicorp's go-plugin package To keep consistency between imports in the plugin package. * test: fix issue that killed plugins when testing * test: fix plugin command tests * fix: correct issue in plugin load * chore: update plugin documentation * chore: fix changelog * chore: add buf to tool dependencies * feat: add make file targets to generate code from proto * fix: correct proto file linting issue * chore: proto file format fix * ci: add GitHub workflow to check and lint proto files * ci: disable proto breaking change It must be disabled until proto files are available in the main branch. * fix: correct Buf repository name * test: change plugin integration test to use a local example folder This was done to allow CI to pass the test without the requirement of merging the changes into the example plugin repository. * chore: update Go and CLI versions Co-authored-by: Danilo Pantani <[email protected]> * ci: update GitHub workflows to use Go version 1.20 * fix: beam me up Scotty * test: move command and manifest test to the right location * test: add plugin gRPC types tests * ci: correct CI issue with naked return * fix: correct plugin command path issue * chore: remove deprecated replace from Go mod Co-authored-by: Julien Robert <[email protected]> * chore: change Ignite App handshake config Co-authored-by: Danilo Pantani <[email protected]> * feat: add bidirectional communication to plugin system (#3544) * chore: rename `types.proto`to `interface.proto` * feat: add code analizer support Adds initial analizer support to allow bidirectional communication between the CLI and the plugins. The analizer features should be defined and implemented on top of these changes. * feat: add analizer support to plugin's interface * chore: add proto generated code * chore: generate interface mocks * refactor: change protocol implementation to support analizer * chore: fix issue with missing return * refactor: change plugin command to add analizer to calls * feat: add proto types required for the cosmos/proto analysis packages These types allows the implementation of the dependencies analyzer. * chore: update plugin template to include analizer arguments * fix: correct type in Analyzer name * chore: update plugin documentation * chore: rename analyzer file name because of typo * fix: correct typo for analyzer names * chore: update changelog * test: fix plugin integration test * test: fix plugin cmd unit tests * test: fix plugin service unit test * ci: fix unused argument warning * refactor: Add basic GetChainInfo method to plugin API (#3561) * refactor: Analyzer/analizer -> ClientAPI * refactor: rename proto files and rebuild * refactor: Add json tags * wip/refactor: Module analysis * feat: Add chain reference to plugin ClientAPI * feat: Complete Dependencies ClientAPI method * fix: Address review comments * feat: Remove services/chain dep from pkg/cosmosanalysis as per discussion * wip: remove deptools install * feat: package-specific includes * fix: Replace Module List call with Chain Info call * chore: Remove chain analysis code * feat: ChainInfo API example template * chore: Update template cli reference * chore: clean up PR * fix: Address review comments * fix: address review comments * fix: address review comments * fix: Tests and linting * chore: add changelog * fix: linting issues * tests: fix issue with client api in plugin tests * tests: fix plugin template for integration tests --------- Co-authored-by: jeronimoalbi <[email protected]> * chore: correct typos and simplify code * chore: update pseudo version for plugin's `go.mod` template * chore: fix typos * fix: correct call to module dependency resolution * chore: remove proto file comment --------- Co-authored-by: Danilo Pantani <[email protected]> Co-authored-by: Clockwork <[email protected]> --------- Co-authored-by: Danilo Pantani <[email protected]> Co-authored-by: Julien Robert <[email protected]> Co-authored-by: Clockwork <[email protected]>
ignite · Nov 10, 2023 · 718f27c · 718f27c
1 parent a031fb3
commit 718f27c
Show file tree

Hide file tree

Showing 59 changed files with 5,093 additions and 1,126 deletions.
diff --git a/.github/workflows/proto-checker.yml b/.github/workflows/proto-checker.yml
@@ -0,0 +1,31 @@
+name: Protobuf Files
+
+on:
+  pull_request:
+    paths:
+      - "proto/**"
+
+permissions:
+  contents: read
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - uses: actions/checkout@v3
+      - uses: bufbuild/[email protected]
+      - uses: bufbuild/buf-lint-action@v1
+        with:
+          input: "proto"
+
+  # TODO: Uncomment after PR#3529 is merged
+  # break-check:
+  #   runs-on: ubuntu-latest
+  #   steps:
+  #     - uses: actions/checkout@v3
+  #     - uses: bufbuild/[email protected]
+  #     - uses: bufbuild/buf-breaking-action@v1
+  #       with:
+  #         input: "proto"
+  #         against: "https://github.com/${{ github.repository }}.git#branch=${{ github.event.pull_request.base.ref }},ref=HEAD~1,subdir=proto"
diff --git a/.gitignore b/.gitignore
@@ -3,7 +3,6 @@ scripts/**/nodetime-*
 dist/
 node_modules
 .DS_Store
-apps/
 .idea
 .vscode
 docs/.vuepress/dist

diff --git a/Makefile b/Makefile
@@ -59,6 +59,26 @@ lint:
 
 .PHONY: govet format lint
 
+## proto-all: Format, lint and generate code from proto files using buf.
+proto-all: proto-format proto-lint proto-gen
+
+## proto-gen: Run buf generate.
+proto-gen:
+	@echo Generating code from proto...
+	@buf generate --template ./proto/buf.gen.yaml --output ./
+
+## proto-format: Run buf format and update files with invalid proto format>
+proto-format:
+	@echo Formatting proto files...
+	@buf format --write
+
+## proto-lint: Run buf lint.
+proto-lint:
+	@echo Linting proto files...
+	@buf lint
+
+.PHONY: proto-all proto-gen proto-format proto-lint
+
 ## test-unit: Run the unit tests.
 test-unit:
 	@echo Running unit tests...

diff --git a/buf.work.yaml b/buf.work.yaml
@@ -0,0 +1,3 @@
+version: v1
+directories:
+  - proto
diff --git a/changelog.md b/changelog.md
@@ -4,6 +4,17 @@
 
 ### Features
 
+- [#3544](https://github.com/ignite/cli/pull/3544) Add bidirectional communication to plugin system
+- [#3561](https://github.com/ignite/cli/pull/3561) Add GetChainInfo method to plugin system API
+
+### Changes
+
+- [#3529](https://github.com/ignite/cli/pull/3529) Refactor plugin system to use gRPC
+
+## [`v0.27.1`](https://github.com/ignite/cli/releases/tag/v0.27.1)
+
+### Features
+
 - [#3476](https://github.com/ignite/cli/pull/3476) Use `buf.build` binary to code generate from proto files
 - [#3614](https://github.com/ignite/cli/pull/3614) feat: use DefaultBaseappOptions for app.New method
 - [#3536](https://github.com/ignite/cli/pull/3536) Change app.go to v2 and add AppWiring feature

diff --git a/docs/docs/apps/02-developing-apps.md b/docs/docs/apps/02-developing-apps.md
@@ -43,31 +43,35 @@ All apps must implement a predefined interface:
 ```go title=ignite/services/plugin/interface.go
 type Interface interface {
 	// Manifest declares app's Command(s) and Hook(s).
-	Manifest() (Manifest, error)
+	Manifest(context.Context) (*Manifest, error)
 
 	// Execute will be invoked by ignite when an app Command is executed.
 	// It is global for all commands declared in Manifest, if you have declared
 	// multiple commands, use cmd.Path to distinguish them.
-	Execute(cmd ExecutedCommand) error
+	// The ClientAPI argument can be used by plugins to get chain app analysis info.
+	Execute(context.Context, *ExecutedCommand, ClientAPI) error
 
 	// ExecuteHookPre is invoked by ignite when a command specified by the Hook
 	// path is invoked.
 	// It is global for all hooks declared in Manifest, if you have declared
 	// multiple hooks, use hook.Name to distinguish them.
-	ExecuteHookPre(hook ExecutedHook) error
+	// The ClientAPI argument can be used by plugins to get chain app analysis info.
+	ExecuteHookPre(context.Context, *ExecutedHook, ClientAPI) error
 
 	// ExecuteHookPost is invoked by ignite when a command specified by the hook
 	// path is invoked.
 	// It is global for all hooks declared in Manifest, if you have declared
 	// multiple hooks, use hook.Name to distinguish them.
-	ExecuteHookPost(hook ExecutedHook) error
+	// The ClientAPI argument can be used by plugins to get chain app analysis info.
+	ExecuteHookPost(context.Context, *ExecutedHook, ClientAPI) error
 
 	// ExecuteHookCleanUp is invoked by ignite when a command specified by the
 	// hook path is invoked. Unlike ExecuteHookPost, it is invoked regardless of
 	// execution status of the command and hooks.
 	// It is global for all hooks declared in Manifest, if you have declared
 	// multiple hooks, use hook.Name to distinguish them.
-	ExecuteHookCleanUp(hook ExecutedHook) error
+	// The ClientAPI argument can be used by plugins to get chain app analysis info.
+	ExecuteHookCleanUp(context.Context, *ExecutedHook, ClientAPI) error
 }
 ```
 
@@ -76,35 +80,34 @@ the method's body.
 
 ## Defining app's manifest
 
-Here is the `Manifest` struct:
-
-```go title=ignite/services/plugin/interface.go
-type Manifest struct {
-	Name string
-
-	// Commands contains the commands that will be added to the list of ignite
-	// commands. Each commands are independent, for nested commands use the
-	// inner Commands field.
-	Commands []Command
-
-	// Hooks contains the hooks that will be attached to the existing ignite
-	// commands.
-	Hooks []Hook
-
-	// SharedHost enables sharing a single app server across all running instances
-	// of an app. Useful if an app adds or extends long running commands
-	//
-	// Example: if an app defines a hook on `ignite chain serve`, a server is instanciated
-	// when the command is run. Now if you want to interact with that instance from commands
-	// defined in that app, you need to enable `SharedHost`, or else the commands will just
-	// instantiate separate app servers.
-	//
-	// When enabled, all apps of the same `Path` loaded from the same configuration will
-	// attach it's gRPC client to a an existing gRPC server.
-	//
-	// If an app instance has no other running app servers, it will create one and it
-	// will be the host.
-	SharedHost bool `yaml:"shared_host"`
+Here is the `Manifest` proto message definition:
+
+```protobuf title=proto/ignite/services/plugin/grpc/v1/types.proto
+message Manifest {
+  // App name.
+  string name = 1;
+
+  // Commands contains the commands that will be added to the list of ignite commands.
+  // Each commands are independent, for nested commands use the inner Commands field.
+  bool shared_host = 2;
+
+  // Hooks contains the hooks that will be attached to the existing ignite commands.
+  repeated Command commands = 3;
+
+  // Enables sharing a single app server across all running instances of an Ignite App.
+  // Useful if an app adds or extends long running commands.
+  //
+  // Example: if an app defines a hook on `ignite chain serve`, a server is instanciated
+  // when the command is run. Now if you want to interact with that instance
+  // from commands defined in that app, you need to enable shared host, or else the
+  // commands will just instantiate separate app servers.
+  //
+  // When enabled, all apps of the same path loaded from the same configuration will
+  // attach it's RPC client to a an existing RPC server.
+  //
+  // If an app instance has no other running app servers, it will create one and it
+  // will be the host.
+  repeated Hook hooks = 4;
 }
 ```
 
@@ -132,16 +135,16 @@ For instance, let's say your app adds a new `oracle` command to `ignite
 scaffold`, then the `Manifest` method will look like :
 
 ```go
-func (app) Manifest() (plugin.Manifest, error) {
-	return plugin.Manifest{
+func (app) Manifest(context.Context) (*plugin.Manifest, error) {
+	return &plugin.Manifest{
 		Name: "oracle",
-		Commands: []plugin.Command{
+		Commands: []*plugin.Command{
 			{
 				Use:   "oracle [name]",
 				Short: "Scaffold an oracle module",
 				Long:  "Long description goes here...",
 				// Optionnal flags is required
-				Flags: []plugin.Flag{
+				Flags: []*plugin.Flag{
 					{Name: "source", Type: plugin.FlagTypeString, Usage: "the oracle source"},
 				},
 				// Attach the command to `scaffold`
@@ -156,14 +159,21 @@ To update the app execution, you have to change the `Execute` command. For
 example:
 
 ```go
-func (app) Execute(cmd plugin.ExecutedCommand) error {
+func (app) Execute(_ context.Context, cmd *plugin.ExecutedCommand, _ plugin.ClientAPI) error {
 	if len(cmd.Args) == 0 {
 		return fmt.Errorf("oracle name missing")
 	}
+
+	flags, err := cmd.NewFlags()
+	if err != nil {
+		return err
+	}
+
 	var (
 		name      = cmd.Args[0]
-		source, _ = cmd.Flags().GetString("source")
+		source, _ = flags.GetString("source")
 	)
+
 	// Read chain information
 	c, err := getChain(cmd)
 	if err != nil {
@@ -199,10 +209,10 @@ resulting in `post` and `clean up` not executing.
 The following is an example of a `hook` definition.
 
 ```go
-func (app) Manifest() (plugin.Manifest, error) {
-	return plugin.Manifest{
+func (app) Manifest(context.Context) (*plugin.Manifest, error) {
+	return &plugin.Manifest{
 		Name: "oracle",
-		Hooks: []plugin.Hook{
+		Hooks: []*plugin.Hook{
 			{
 				Name:        "my-hook",
 				PlaceHookOn: "ignite chain build",
@@ -211,8 +221,8 @@ func (app) Manifest() (plugin.Manifest, error) {
 	}, nil
 }
 
-func (app) ExecuteHookPre(hook plugin.ExecutedHook) error {
-	switch hook.Name {
+func (app) ExecuteHookPre(_ context.Context, h *plugin.ExecutedHook, _ plugin.ClientAPI) error {
+	switch h.Hook.GetName() {
 	case "my-hook":
 		fmt.Println("I'm executed before ignite chain build")
 	default:
@@ -221,8 +231,8 @@ func (app) ExecuteHookPre(hook plugin.ExecutedHook) error {
 	return nil
 }
 
-func (app) ExecuteHookPost(hook plugin.ExecutedHook) error {
-	switch hook.Name {
+func (app) ExecuteHookPost(_ context.Context, h *plugin.ExecutedHook, _ plugin.ClientAPI) error {
+	switch h.Hook.GetName() {
 	case "my-hook":
 		fmt.Println("I'm executed after ignite chain build (if no error)")
 	default:
@@ -231,8 +241,8 @@ func (app) ExecuteHookPost(hook plugin.ExecutedHook) error {
 	return nil
 }
 
-func (app) ExecuteHookCleanUp(hook plugin.ExecutedHook) error {
-	switch hook.Name {
+func (app) ExecuteHookCleanUp(_ context.Context, h *plugin.ExecutedHook, _ plugin.ClientAPI) error {
+	switch h.Hook.GetName() {
 	case "my-hook":
 		fmt.Println("I'm executed after ignite chain build (regardless errors)")
 	default:
-Original file line number
+Diff line change
@@ Expand Up / @@ -3,7 +3,6 @@ scripts/**/nodetime-* @@
     dist/
     node_modules
     .DS_Store
-    apps/
     .idea
     .vscode
     docs/.vuepress/dist
@@ Expand Down @@