Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] 能否把eventesh作为消息中心使用? #4823

Open
1 task done
9997766 opened this issue Apr 10, 2024 · 6 comments
Open
1 task done

[Question] 能否把eventesh作为消息中心使用? #4823

9997766 opened this issue Apr 10, 2024 · 6 comments
Labels
question Further information is requested Stale

Comments

@9997766
Copy link

9997766 commented Apr 10, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

Question

我想搭建一个evenemesh集群,用nginx作为负载均衡,应用系统集成eventmesh-sdk,访问nginx,连到eventmesh。这个是是否能做到,推荐的部署结构是什么?

我希望evenemesh每个节点是无状态的,不管应用系统通过nginx连到哪个节点上,都能流畅的使用http tcp grpc 消费消息。重启应用服务,或者重启eventmesh 不会造成消息丢失。

我试了用rabbitmq作为存储,是不能达到这个效果的,因为rabbitmq-client.properties中,只能配一个队列。如果集群中任何一个eventmesh消费了消息,其他的eventmesh是无法接收该消息了,自然也无法返回给应用系统。比如应用通过nginx连到eventmesh节点1上,节点1消费了消息,节点2的evenmesh就无法收到该消 , 就无法推给连节点2上的应用系统。

上面的效果是否能做到?使用哪种部署结构能搞定?

@9997766 9997766 added the question Further information is requested label Apr 10, 2024
@Pil0tXia
Copy link
Member

用nginx作为负载均衡,应用系统集成eventmesh-sdk,访问nginx,连到eventmesh。这个是是否能做到

Yes, you can. You need to use nginx as the load balancer of multiple EventMesh Runtime instances, and reverse proxy Runtime's traffic. In this case the SDK client will establish a subscription on one Runtime instance, and the specific Runtime will upload the subscription info to Meta (known as the Registry like nacos).

However, if the Runtime responsible for this subscription goes offline, EventMesh's load balancing mechanism will allow other Runtimes to take over subsequent messages for this subscription, but nginx is not aware of this change.

我试了用rabbitmq作为存储,只能配一个队列

We recommand RocketMQ as Event Store.

如果集群中任何一个eventmesh消费了消息,其他的eventmesh是无法接收该消息了,自然也无法返回给应用系统

Only one EventMesh Runtime instance is responsible for a subscription at the same time in a cluster. You may need to link two sink client to the same Runtime instance.

@9997766
Copy link
Author

9997766 commented Apr 11, 2024

请问eventmesh具体是什么定位,如果定位为应用系统的sidecar,每个应用节点需要配一个eventmesh节点,比较重
但如果作为集群使用,eventmesh 不支持集群。

其他问题回复:
We recommand RocketMQ as Event Store.

用nginx作为负载均衡,应用系统集成eventmesh-sdk,访问nginx,连到eventmesh。这个是是否能做到

Yes, you can. You need to use nginx as the load balancer of multiple EventMesh Runtime instances, and reverse proxy Runtime's traffic. In this case the SDK client will establish a subscription on one Runtime instance, and the specific Runtime will upload the subscription info to Meta (known as the Registry like nacos).

However, if the Runtime responsible for this subscription goes offline, EventMesh's load balancing mechanism will allow other Runtimes to take over subsequent messages for this subscription, but nginx is not aware of this change.

我试了用rabbitmq作为存储,只能配一个队列

We recommand RocketMQ as Event Store.
RocketMQ 太不稳定了。不知道是eventmesh的原因,还是rocketmq的原因,发现各种log各种状态不同步、netty连接不上、连接不上之类的问题,没法用。 我在尝试试用kafka,看看有没有问题。

如果集群中任何一个eventmesh消费了消息,其他的eventmesh是无法接收该消息了,自然也无法返回给应用系统

Only one EventMesh Runtime instance is responsible for a subscription at the same time in a cluster. You may need to link two sink client to the same Runtime instance.
那就是不支持集群了,如果这个节点挂了,其他节点无法无感的接到其他的节点上。

@9997766
Copy link
Author

9997766 commented Apr 11, 2024

我看rocketmq 和 kafka,是支持广播性消费的。当eventmesh作为消费者,向rocketmq 和 kafka订阅消息,是否可以略微调一下代码,支持广播式的接收消息,这样应用系统不管接到那个eventmesh节点上,就都可以消费消息了。

@xwm1992
Copy link
Contributor

xwm1992 commented Apr 11, 2024

我看rocketmq 和 kafka,是支持广播性消费的。当eventmesh作为消费者,向rocketmq 和 kafka订阅消息,是否可以略微调一下代码,支持广播式的接收消息,这样应用系统不管接到那个eventmesh节点上,就都可以消费消息了。

应用系统与eventmesh建立连接后,再次向对应的eventmesh发起订阅,eventmesh才会启动对应MQ客户端进行消费,你说的不支持集群我不是很理解,只要应用系统可以在连接断开后重定向到可用的eventmesh上即可,与广播消费模式没关系,不清楚你为什么得出eventmesh不支持集群模式的结论。

@9997766
Copy link
Author

9997766 commented Apr 11, 2024

我看rocketmq 和 kafka,是支持广播性消费的。当eventmesh作为消费者,向rocketmq 和 kafka订阅消息,是否可以略微调一下代码,支持广播式的接收消息,这样应用系统不管接到那个eventmesh节点上,就都可以消费消息了。

应用系统与eventmesh建立连接后,再次向对应的eventmesh发起订阅,eventmesh才会启动对应MQ客户端进行消费,你说的不支持集群我不是很理解,只要应用系统可以在连接断开后重定向到可用的eventmesh上即可,与广播消费模式没关系,不清楚你为什么得出eventmesh不支持集群模式的结论。

很简单,2个点:

1 ) eventmesh 应该是无状态的,如果任何一个节点失败,其他节点应能无感接管应用系统的链接,而不是需要应用系统重新链接。像grpc、tcp 这种长链接,因为网络之类的原因,导致长链接断开很正常,如果每次需要应用系统重新订阅,非常麻烦。而且jdk中目前也无此判断机制。

2) 目前的机制,如果eventmesh节点失败,应用连接到其他节点上,因为状态不一样,要么丢消息、要么重复消费。

此类的机制,可以了解一下nacos基于grpc的长链接管理机制。 应用系统断开和某个nacos节点的链接,重新链接其他nacos节点,不会产生数据不一致的情况。我理解是eventmesh 从技术架构上,尚无状态同步的机制。

@Pil0tXia Pil0tXia changed the title 能否把eventesh作为消息中心使用? [Question] 能否把eventesh作为消息中心使用? Apr 12, 2024
Copy link
Contributor

It has been 90 days since the last activity on this issue. Apache EventMesh values the voices of the community. Please don't hesitate to share your latest insights on this matter at any time, as the community is more than willing to engage in discussions regarding the development and optimization directions of this feature.

If you feel that your issue has been resolved, please feel free to close it. Should you have any additional information to share, you are welcome to reopen this issue.

@github-actions github-actions bot added the Stale label Jul 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested Stale
Projects
None yet
Development

No branches or pull requests

3 participants