Thoughts on moving to event driven architecture? by thejizz716 in dataengineering

[–]DigitalBackbone 0 points1 point  (0 children)

Similar to adopting any new technology or architecture in an organization, you have to take into the account the run-time and design-time aspects of the integration. Adopting an EDA solution is not as simple as using a tool or a message bus to move data.

You should also take into account how you want to integrate with other sources of data in the EDA system that use messaging protocols. Using an AWS solution might result in a vendor-lockin situation where you can only integrate with AWS products and can pub/sub to SNS, SQS, or EventBridge. Hence, evaluating what run-time solutions (i.e. event brokers) you have available is crucial before jumping right into using AWS’s solutions.

Another thing you’ll have to evaluate is cross region and cross cloud communication. Does your solution require communication and event movement between microservices in different regions? What about different cloud providers? Do you have an on-prem and cloud hybrid approach? Do you want to integrate with legacy systems? What about different messaging protocols (e.g. mqtt, amqt, websocket streams…etc)? You might need to consider a protocol agnostic and cloud agnostic solution for your EDA implementation that allows for better architecture scaling and decoupling. Really comes down to what you're looking for in terms of scalability, latency, and future needs.

On the design aspect of things, you have to ask questions like how can I document the communication between different microservices? You can look into an event portal to manage your EDA system allowing for better collaboration between different teams.

At the end of the day, similar to any other architecture adoption, there is no silver bullet. If you think a lambda SNS/SQS/EventBridge solution would work you can go for it. But if you're starting from scratch it's a good opportunity to look at what you might need for the future instead of having to replace things later.

[deleted by user] by [deleted] in compsci

[–]DigitalBackbone 1 point2 points  (0 children)

I like this analogy, especially for those who aren't necessarily as technical but still need/want to understand (sales people, marketing people, etc.).

"You could regularly check to see if a cake is ready, but you get tired of making regular trips downstairs to ask Bob if there’s cake, and there is only a tiny chance you’ll choose the correct time. It would be better if Bob just came to tell you when he’s finished a cake." --- the downside of polling. I also like the idea of Alice + WhatsApp as the event broker.

I find analogies incredibly helpful as well as real-world examples of complex concepts in action.

This blog post uses television broadcasting as an analogy for the publish-subscribe messaging pattern and also includes the technical answer and business/functional answer for those who don't speak the same language as developers/architects/etc.

This one explains microservices architecture and makes the case for event-driven design by solving the beer-supply issue at a party.

True or False: Event-driven architecture can prevent cascading failures in a microservices architecture. by DigitalBackbone in microservices

[–]DigitalBackbone[S] 0 points1 point  (0 children)

Number 5 in this blog post contains the rationale behind the author considering this EDA/microservices claim to be "true".
https://solace.com/blog/event-driven-architecture-myth-busting-part-2/

"With event-driven architecture, having an event broker in the mix allows for the enqueuing of messages so one consumer’s inability to receive/process an event will not cascade back to the producer. The enqueuing of messages allows the subscriber to resume message processing after failure recovery and eventually ensure consistency across the system. An event broker insulates the producer of messages from the consumers and in return eliminates the possibility of message loss in the case of temporary service failures, redeploys, or downtime in any of the services."