[Question] Shared integration events

[AlexanderSysoev]

In your application integration events has duplicate in several microservices. For example UserCheckoutAcceptedIntegrationEvent is located in Basket.API and Ordering.API. Would it be the best to share contracts between microservices (place integration events in separate assembly)?

[ardalis]

The goal for microservices is that they be self-contained, and not have to be updated based on changes in other projects or services. Thus, some duplication is desirable in this case if it means that changes in one microservice do not need to impact the others.

[CESARDELATORRE]

@ AlexanderSysoev - Great question! 👍
@ardalis - Agree. Unless it is about NuGet packages with re-usable libraries "agnostic" of the project, such as cross-cutting-concerns like a library for handling JSON, or a library for handling Http resilient communication (Polly, etc.), data structures, DTOs, etc. data structures, code or DTOs should not be shared across microservices as they could be developed by completely independent development teams.

THE POINT: Share between microservices only what you would also share between different/independent applications. 👍

However, In the specific case of Integration Events I should also say that some developers prefer to share the types/libraries between microservices. In this case we chose to be more independent between microservices, so not to couple multiple microservices to a single "event's library".

[mvelosop]

Hi @ardalis, @CESARDELATORRE

I know we've talked about this more than once, but I'm hanging on @CESARDELATORRE's last paragraph.

I think that there IS A REAL AND IMPORTANT dependency between microservices that share integration events, because those teams at some point reached an agreement to settle on that information exchange spec (it's a contract after all), so I think it's actually good to have that dependency expressed in code somehow, and a shared library looks like a good option.

Am I way off?

[CESARDELATORRE]

My point is that for microservices integration you should take a similar approach than for application integration. Both microservices and application development must be independent/autonomous.
It could be that initially those teams are collaborating, but it could also be the case where multiple teams don't have such a close collaboration in the same way that you could publish HTTP services for external application integration and you don't provide a library/assembly with the DTOs or Events because that would cause a strong dependency between the services and the external applications.
I think it should be the same case for microservices. They must not share "business" assemblies, only cross-cutting concerns NuGet packages of libraries that you'd treat like external NuGet packages.

But, as always, this decision must depend on the application's context.

[mvelosop]

Fair enough!

[Carl-Hugo]

In the case each microservice has its own copy of an integration event:

1. If one microservice change an event, get deployed, then publish it.
2. All other microservices subscribed to (relying on) that event will (or could) break.

So, as mentioned by @mvelosop there was, at some point, an agreement that created a contract.

That leads me to the following conclusion (feel free to add to it, by agreeing or disagreeing):

• Sharing the contracts using an assembly creates a dependency on that assembly.
  • This could be shared and versioned using a NuGet package (at an extent it could also be used to manage multiple version of the same events).
  • That same assembly becomes the source of truth for events stored in it.
  • This assembly becomes a catalog of events, making the event discovery easier.
  • Contracts could even be enforced at compile time (in a CI/CD pipeline for example).
• While copying all classes:
  • Hides those dependencies
  • Makes the discovery of those dependencies harder
  • Does not add anything (I can think of some edge cases where it could add a little)

So whatever the technique used, the messages (events) still need to be in the same format (or at least be similar) in all microservices (publishers and subscribers), so the dependency is there no matter what (hidden or not).

Personally, for most scenarios, I think I'd go for discoverability and code sharing (DRY) over "falsy decoupling" and copy/paste.

Finally, as stated by @CESARDELATORRE : "But, as always, this decision must depend on the application's context."

[mehdipayervand]

@AlexanderSysoev Consider each microservice would be developed by different language and different platform for example python or java.
as it is RabbitMQ and REST both have their own implementation so one team have Python skill and experience and so on.

[mvelosop]

Closing this discussion now (it was a nice - and recurring - one), will reopen if needed.

[dgrandemange]

Interesting topic indeed.
As a very newcomer to microservice architecture, i read a lot from Microsoft dotnet documentation on the subject (which i found great by the way).
I understood very quickly the loosely coupled micro-services purpose of such architecture.
But then, reading the Implementing event-based communication between microservices topic, and the integration events produced by some micro-services while consumed by other subscriber micro-services, i felt lost because somehow there was coupling there.
Well, now, reading this topic, i just feel like : ok, integration events need their respective publishers to document them, not under the form of physical dependencies that would tie to a specific technology, but rather through a contract documentation (HTML, markdown, whatever) made available to any potential subscribers interested. As if we were exposing and documenting a RESTful API.
I understand the AsyncAPI initiative is going this way (as OpenAPI is doing with REST). This project is actively looking for contributors.

[mvelosop]

Hi @dgrandemange, thanks for the link, it looks quite interesting to take a detailed look at 👍

BTW, you might want to take a look at the new centralized logging feature we have now, with a nice example on integration events handling.

Hope this helps.

[alexanderbikk]

Hi
Very interesting discussion. I also investigated the UserCheckoutAcceptedIntegrationEvent and was wondering about shared library. My team works only with .Net for now and I thought that shared library would be good solution(according to pros described above)

But I also imagine the situation when we have 3 microservices for example A, B, C.
The service A have one contract models with service B and two other contract models with service C(according to business requirements).
So in this case I should create two shared libraries(AB, AC), since if I create only one shared library I will face with problem when I need deploy all microservices when only the contract between A and B services was changed. So I should have one shared library per to services that communicate between each other, looks like not very good approach if I will have more than 3 microservices (too many shared libraries).

Correct me if I'm wrong.

Thanks

[AlexanderSysoev]

@alexanderbikk I believe that contract belongs to concrete microservice, it is not a "proxy" between them. According to this microservices B and C exposes their own contracts. If microservice A need to interact with B and C you should reference B and C contracts in A. When interaction between A and B changes, you change contract B and update microservices A and B only.

[alexanderbikk]

@AlexanderSysoev ok, I understand. So, I should have separate library with contracts for each service(if I choose approach with shared libraries). Am I right?

[AlexanderSysoev]

@alexanderbikk you are right

[mvelosop]

Hi @alexanderbikk, @AlexanderSysoev,

I also thought that sharing a library on integration events was something good, and even argued with @CESARDELATORRE about that, as you might have seen above.

But that was seven months ago!

However, while implementing the logging system, I realized that even though integration events have the same name in different microservices, they have different properties in each one (at least some of them), to handle only the properties each microservice really cares about.

Then I realized that, for example, one team might find it useful to implement a calculated property that adds value in a microservice, but would be confusing somewhere else, so that property should not "leak" out to others.

So, nowadays I agree 100% with @CESARDELATORRE on this subject.

Hint, the guy knows a bit 😉.

[alexanderbikk]

Hi @mvelosop. I also thought about case that you described when we have different properties in contracts in different microservices. It allows more flexibility during deployment. So I deploy only microservice where the contract was changed and don't touch the other services that also using this contract.

Thank you, guys

[nemanjarogic]

Hi @CESARDELATORRE , @mvelosop.

After some time thinking about shared integration events I completely understand your reasons for not sharing "business" assemblies, and sharing only cross-cutting concerns (building blocks).

However, what should we do if we have multiple microservices following DDD (like Ordering in eShopOnContainers)? Where should we place Entity, ValueObject, IAggregateRoot, IRepository and similar classes? Should we share these classes among multiple microservices or every microservice should have its own implementation?

These classes don't deal with some business concerns, but still I'm not sure about the right place for them.

[mvelosop]

Hi @nemanjarogic,

I think, at least as of today 😉, that it's mainly a matter of team size.

Or perhaps more about team dynamics, if it's easy for your team to agree on code to share, then do it and consider this as you would any other "general-use" package like MediatR or whatever.

I you start feeling like you're loosing too much time arguing, then don't share the code now. This is not "bad" either, it can help you find a better overall solution, based on real usage.

BTW, my general approach to this now is, I don't share anything out from the start. When actual usage patterns emerge, then I begin to consider refactoring the common parts out to a library, but the code has to "show its worth in actual battles" to achieve such a status.

I just lost the count of one-use libraries I created 😅

Hope this helps.

[agvglobant]

I think everybody agrees that publishing / consuming events in a microservice creates an integration contract. I also believe everybody thinks making these contracts / dependencies explicit is a good thing, while also keeping your microservice technically decoupled from other microservices.
Note that I said "technically", because (as someone already mentioned) there is always the risk of a breaking change impacting an existing microservice.

So, my take here is to treat these contracts as any other contract.

Wherever I have a contract (owned or consumed) I need to have some VERSIONING mechanism in place, and some EXPLICIT versioning POLICY for producers and consumers. For example, defining a "standard" way for an event to include some contract version information (like a "version field" or something like that), defining that a breaking change makes mandatory changing the major version of the contract, and so on. How the consumer service would react to a missing field, or an additional field it don't recognizes.

So, in the case of integration events, if your application needs to live in an environment where integration contracts might change (or change a lot) maybe is valuable to implement some extra protections for those cases.

Complexity has a cost.... and a good tradeoff is always valuable.

Perfect is the enemy of Good....

Agv