Using events in a system is great, but how do you know for sure if you’ve reliably dispatched your events? The transportation of events needs to be done reliably while maintaining overall system consistency, be it eventual or immediately. In a typical setup, a database is used to store information and queues are used to send messages between processes and systems. Often, the events are dispatched directly to the queue in the same action that stores information in the data. This may not seem to be obviously wrong, but there is a potential problem with this approach.

Non-transactional dispatching

When events are dispatched directly to the queue, there are two network requests happening in a single request or operation, and that creates a problem. Two network requests can not be made atomic (to fail or succeed together).

To illustrate, here is a quick rundown of what might happen:

1. Store data first, then dispatch the message
   • If storing the data fails the operation is failed, unfortunate but not causing inconsistency.
   • If storing the data is OK but publishing events fails the outside world is not kept up to date.
2. Dispatch the message, then store the data (never do this)
   • If dispatching the message fails the operation is failed, unfortunate but not causing inconsistency. But ask yourself, would you be OK with rolling back just because of this?
   • If dispatching the event is OK but storing the data fails the outside world is AHEAD of the internal consistency (this is madness).

The computer science laws literally limit our options and we need to rethink our approach. Even though the likelihood of this happening can be reduced by using retries, you can never fully prevent it. At a certain scale, these kinds of inconsistencies can and will hurt you. Let’s see how transactional outboxes can help.

Transactional dispatching

Our goal is to make the dispatching of events fail or succeed together with storing the application state. To do this, we need to make sure events are dispatched in the same operation used to store state in the database. This means we need to store the events in the database, in a buffer table. By doing so, we’re effectively using the database as a queue. Now I can already hear you say “but using a database as a queue is bad”, and generally I would agree with you. Using a database as a queue is generally not a good idea, so why is it