Dual-write problem
Writing one change to two systems, such as a database and a broker or an old and a new store, without a shared transaction, so they can drift apart.
The code looks innocent: save the booking, then publish “visit booked”, or during a migration write to the legacy database and to the new one. If the process crashes between the two writes, the network times out or the second write is rejected, the systems disagree, and nothing tells you. Retrying does not fix it, because you may not know which write happened. Confluent walks through the failure cases.
The fixes all make one write the truth and derive the other. A transactional outbox stores the event in the same transaction and sends it later. Change data capture reads the database log and publishes from it. During a strangler fig migration, pick one system of record per piece of data and sync the other side from it, then flip the direction when the new system takes over.
Sources
Related terms
- Transactional outbox: Saving an event to an outbox table in the same database transaction as the change itself, then publishing it to the broker separately.
- CDC (change data capture): Reading committed changes from a database’s own transaction log and streaming them as events to other systems.
- System of record (source of truth): The one system whose copy of a piece of data wins when copies disagree; all other copies are derived from it.
- Strangler fig pattern: Replacing a legacy system piece by piece, routing one feature at a time to new code until the old system can be switched off.
- Eventual consistency: A guarantee that copies of data will agree once updates stop arriving, but may briefly show different values in the meantime.
