A lightable full-featured polyglot capability support

[qqeasonchen]

gRPC is a modern open source high performance Remote Procedure Call (RPC) framework that can run in any environment. It can automatically generate idiomatic client and server stubs for your service in a variety of languages and platforms,which can help eventmesh to supports as many client languages as gRPC do (Java, Go, C++, Python, etc...).
here is the link :
https://github.com/grpc/grpc-java/tree/master/netty

Reactive gRPC is a suite of libraries for using gRPC with Reactive Streams programming libraries. Using a protocol buffers compiler plugin, Reactive gRPC generates alternative gRPC bindings for each reactive technology. The reactive bindings support unary and streaming operations in both directions. Reactive gRPC also builds on top of gRPC's back-pressure support, to deliver end-to-end back-pressure-based flow control in line with Reactive Streams back-pressure model.

Reactive gRPC supports the following reactive programming models:

RxJava 2
Spring Reactor

here is the link : https://github.com/salesforce/reactive-grpc

[msdhinesh-geek]

@qqeasonchen , I like to involve into this feature , let me come up with analysis for further discussion.

[qqeasonchen]

@msdhinesh-geek sure ok.

[jinrongluo]

I am very interested in this topic about Integrating grpc/reactive-grpc as transport framework in EventMesh Runtime. I would like to contribute as well.

I am studying the following materials

• grpc https://grpc.io/docs/languages/java/quickstart/
  https://github.com/grpc/grpc-java
• reactive-grpc https://github.com/salesforce/reactive-grpc

One thing I would like to bring up when doing this integration: EventMesh runtime is already having two kinds of transport framework: HTTP and TCP. They are both implemented using Java Netty. gRPC is NOT compatible with Java Netty. This means the following design choices needs to consider:

Option 1. Introduce gRPC as a third transport framework. Keep existing HTTP and TCP Netty code in Runtime side and SDK side

Option 2. Replace exiting TCP with gRPC. This means a MAJOR code refactor in Runtime and SDK modules. Existing Netty Server and client code will be removed and replaced with gRPC related codes.

Option 3. Replace BOTH existing HTTP and TCP Netty framework with gRPC. This means a MAJOR code refactor in Runtime and SDK modules. Existing Netty Server and client code will be removed and replaced with gRPC related codes.

In my opinion, Option 3 provide more benefits over Option 1:

• gRPC offers streaming and async APIs which can effectively satisfies the needs of existing Eventmesh TCP and HTTP Pub/Sub APIs. This claim needs to be verified by Prototyping and Proof Of Concept
• gRPC offers multiple languages SDK generator, which can benefit Eventmesh community for having multi-language SDKs
• Having single gRPC transport has much less code maintenance overhead in the long run. Option 1 and Option 2 still maintenance multiple transport codes, and It can be confusing to end user for not knowing which to choose, HTTP or TCP or gRPC? To end user, as long as they can perform Pub/Sub, the transport protocol does not matter.

[vongosling]

#564
#532
#486
#563

IMO, we should refer to all related issues, @jinrongluo would you like to help to make a proposal for this?

[jinrongluo]

Sure for supporting. Let me do some prototype and POC on the reactive-gRPC and then make a design proposal here. Thanks.

[zmt-Eason]

I am studying gRPC recently and I agreed to the opinion of @jinrongluo. Using gRPC instead of TCP and HTTP can not only satisfies the needs of existing Eventmesh TCP and HTTP Pub/Sub APIs but also benefit Eventmesh community for having multi-language SDKs due to its multiple mature languages SDK generator, which are all we need.

[qqeasonchen]

@vongosling @jinrongluo @zmt-Eason glad to see your intrests and let's do it together.

[ruanwenjun]

I am very interested in this topic about Integrating grpc/reactive-grpc as transport framework in EventMesh Runtime. I would like to contribute as well.

I am studying the following materials

• grpc https://grpc.io/docs/languages/java/quickstart/
  https://github.com/grpc/grpc-java
• reactive-grpc https://github.com/salesforce/reactive-grpc

One thing I would like to bring up when doing this integration: EventMesh runtime is already having two kinds of transport framework: HTTP and TCP. They are both implemented using Java Netty. gRPC is NOT compatible with Java Netty. This means the following design choices needs to consider:

Option 1. Introduce gRPC as a third transport framework. Keep existing HTTP and TCP Netty code in Runtime side and SDK side

Option 2. Replace exiting TCP with gRPC. This means a MAJOR code refactor in Runtime and SDK modules. Existing Netty Server and client code will be removed and replaced with gRPC related codes.

Option 3. Replace BOTH existing HTTP and TCP Netty framework with gRPC. This means a MAJOR code refactor in Runtime and SDK modules. Existing Netty Server and client code will be removed and replaced with gRPC related codes.

In my opinion, Option 3 provide more benefits over Option 1:

• gRPC offers streaming and async APIs which can effectively satisfies the needs of existing Eventmesh TCP and HTTP Pub/Sub APIs. This claim needs to be verified by Prototyping and Proof Of Concept
• gRPC offers multiple languages SDK generator, which can benefit Eventmesh community for having multi-language SDKs
• Having single gRPC transport has much less code maintenance overhead in the long run. Option 1 and Option 2 still maintenance multiple transport codes, and It can be confusing to end user for not knowing which to choose, HTTP or TCP or gRPC? To end user, as long as they can perform Pub/Sub, the transport protocol does not matter.

+1, using grpc can make the code clean. It's benefit to maintenance and easy to implement other language sdk.

[jinrongluo]

Hi @qqeasonchen @ruanwenjun

Thanks for your support. I am adding a gRPC design doc and protobuf data model. and generated java codes. Please review the following PR. Let me know if you have any comments.

PR - #593

Next step, I will write the code to support gRPC in Java-SDK and eventmesh-runtime

[jinrongluo]

Due to code conflicts, I create a new PR - #594