Hey!

Summary of this email:

• Why you shouldn't use the database for an Event Bus. (+ published on Codely Pro)

⏱️ Estimated reading time: Just a bit (things get spicy).

❌ Why you shouldn't use the database for an Event Bus

A couple of weeks ago we published a post on LinkedIn discussing Why you shouldn't use RabbitMQ or Kafka for an Event Bus and we didn't expect the debate it would generate.

So we're taking advantage of today's newsletter to respond to some of those points and share our opinion.

🔥 This setup only works if you have a single worker consuming

This one's easy to answer: You can have multiple workers consuming.

The only thing to keep in mind is to use a SELECT FOR UPDATE to lock the rows so we don't process messages in duplicate.

This leads us to the next one:

🔐 Lock and deadlock management

When thinking about a database, this is THE issue. It can happen that if we have more than one worker/consumer doing a SELECT FOR UPDATE on the same table, we end up with these problems.

And if you've ever been on call, getting woken up at 4am by a deadlock alert is one of the worst things there is.

Fortunately, most modern databases have mechanisms to only lock the returned rows rather than the entire table. If we combine that with a timeout for locks on those queries, we can avoid this major problem.

⏱️ If I'm going to spend hours implementing this solution, I'd rather spend them on the proper tool

This is a totally valid thought and it can make sense depending on the context.

If we already have an in-memory event bus and want to implement an asynchronous one, chances are our stack already has a database and not a messaging system.

If that's the case, it looks like it would be simpler and faster to do it in the database itself. If we're a company with a massive load and we can tell the database will fall short for this type of operation, then it does make sense to go straight for "the proper tool."

🔔 It may sound great, but when loads go past a certain threshold it doesn't work

Absolutely right. The proposal of building the event bus in the database is not meant for large scale, it's meant to have a middle ground between "I don't have an event bus" and "I'll set up a Kafka."

The database event bus can easily scale to tens of thousands of events per second. This is probably more than enough for most projects. If we have requirements beyond that, that's the moment to move to another system. But until then, we can postpone the decision.

All of this is what we explain in the Event bus in database course. Build a scalable event bus until you need to migrate to another system.

We want to take this newsletter to say THANK YOU for the community we have.

Getting these responses and seeing this kind of debate come to life fills us with energy. Questioning the things we take for granted is the best way to improve. Thank you, truly.

And since you've made it this far into the newsletter, here's the joke of the week, which I know you've been waiting for:

> Why did the database administrator break up with their partner? Because they had a one-to-many relationship 😂 😂 😂

See you around!