What is the Event Sourcing Pattern? | Microservices 101

hi I'm Wade from confluent traditional database architecture can be thought of as a road trip you might take many twists and turns on your journey but where you are now is all that matters except it isn't the journey is often just as important or more important than the destination but how does that translate into a database architecture traditionally when we store data in a database we do so in a destructive fashion each time we update a record we destroy whatever data was there previously this

means that we lose the history of the events that led to the current state but what if we didn't want to lose the history what if we were interested not just in where we are but also in how we got there have you ever investigated a bug only to discover that the current state doesn't contain enough information for you to diagnose or fix the problem you need to know what specific changes a user made to arrive at that State if you've seen this kind of situation before let let me know in the commen

ts one way to solve this would be to persist an audit log along with the state each time we update a record in the database we can write an audit event to another table these events would contain details about what changed who changed it and why the events would be kept forever because we never know when we might need access to the history this can be very important in highly regulated Industries such as banking or Healthcare however these audit entries do have potential issues if we have implem

ented the log correctly any details that exist in the state will also exist in the log this leads to data duplication and if we have duplicate data then what happens if it gets out of sync ideally we'd perform any updates in a transactional fashion to prevent that but bugs happen and when they do we need a plan to deal with them when our state and audit log are in disagreement ment the safe choice is to rely on the audit log it contains a full history of all the events and therefore is typically

more reliable than the state alone but this begs the question if all of the data is contained in the audit log and if the audit log is the final source of truth then why do we need the state couldn't we just scrap the state and use the audit log Instead This is the basic principle behind event sourcing each time we update an object inside of a microservice we don't store the current state of the object in the database instead we store the events that led to the state essentially the audit entri

es if we need to reconstruct the object then we can replay all of the events and use them to calculate the current state banking is an excellent example of how this works the account balance represents the current state however that's not what the bank stores instead the bank stores a list of all of the transactions that led to the current balance if if we want to increase our balance from $100 to $200 then we do so by depositing an additional $100 rather than storing the New Balance we instead

store an event perhaps named funds deposited and we record important details about the event such as how much was deposited and when later when we want to know the current balance of the account we can replay all of the events to calculate it the advantage of this approach is that if a mistake happens anywhere along the way we can use the historical record to locate the mistake and issue a correction we can even replay portions of the event log to reconstruct what the state was at any time in th

e past if I want to know what my balance was at 300 p.m. last Tuesday all of the information required to answer that question is readily available that wouldn't be true if I was only storing the balance and not the transaction history another key Advantage provided by event sourcing is that it allows us to solve the Dual right problem when we build event-driven systems we often need to record data in our database but also emit events to a secondary system such as Apache Kafka because Kafka and t

he database aren't connected there's no way to update both in a transactional fashion this can leave us in a situation where One update fails and the other succeeds and our data becomes out of sync however using event sourcing we can solve this problem rather than trying to update both the database and CFA at the same time we worry only about the event log remember it's our single source of Truth as long as the event makes it into the log then we consider it to be correct we can then have a sepa

rate process that scans the event log and emits any new events to CFA once this separate process finishes we'll be able to guarantee that the database and CFA are in sync we've eliminated the risk of data loss of course event sourcing isn't a perfect solution it can introduce complexity especially when we need to deal with queries that span multiple data objects tools such as command query responsibility segregation or cqrs can help with this but they come at a cost so although event sourcing ca

n seem like a very powerful tool it may not be the tool we want to reach for in every situation it can be a great option when we're building portions of our system give a competitive advantage or require auditing but for systems without these requirements the added complexity can outweigh any advantages if you want a deeper dive into the Dual WR problem check out the video linked below for more information on event sourcing make sure you look at the course found in confluent developer and on our

YouTube channel don't forget to like share and subscribe and I'll see you next time