hi I'm Wade from confluent let's take a moment and talk about how to represent events in Kafka when working with.net [Music] an event-driven system is made up of many moving Parts the individual pieces often take the form of microservices but that's not a strict requirement you can build event-driven systems using a variety of different architectures these systems are often built on the backbone of Kafka essentially Kafka acts as the central nervous system of the application it facilitates commu
nication between the various pieces a key element of this is that these Services communicate primarily using asynchronous events this asynchronous communication allows each piece of the system to operate with increased autonomy this in turn leads to other benefits such as looser coupling increased reliability and better scalability what do we mean when we call something an event Martin Fowler describes an event as something that has happened in the outside world that is of interest to the applic
ation this can be obvious such as a customer buying an item in a store but it can also be something more technical in nature for example if we have a fitness tracker we might record an event each time the tracker connects to a mobile device in order to import data into our backend Services regardless of what it represents events are always something that happen in the past this has important consequences because back in the present we can't change the event it's considered to be immutable changi
ng a past event would be like inventing time travel and if Hollywood has taught me anything that's not a good idea another consequence is that the events are typically named in past tense for example we would name an event user created rather than create user we also try to be descriptive with the names so that the events reveal their intended effect on events such as user update it doesn't reveal much while an event like user address changed reveals a lot more once we've established what our ev
ent is and what kind of data it will contain the next step is to package it into a message that we can push into Kafka a Kafka message consists of two main parts you can see them represented here as the generic types we have labeled key and value message keys are usually represented by simple types such as strings or integers although they can be more complex types it's not common the actual value of the key can be defined in line when you construct the message its primary purpose is to determin
e how the messages will be distributed in a cluster this in turn impacts the ordering guarantees of those messages if you care about the ordering of your messages you'll want to pay close attention to the key however if the order doesn't matter then the key can be considered optional often the message key will be the identifier of a specific domain entity such as a user ID or perhaps a device ID in our fitness tracker example using the device ID will ensure that all messages for that specific de
vice are handled in order the second part of the message is the value this is where we would typically store the details of our event like with the key it can be a simple type such as a string but often we use a more complex object that can be serialized into formats such as Json Avro or protobuf this might be a representation of our event or it might be a domain entity that is relevant to the event this is the part of the message that most of the downstream consumers are going to be interested
in they will consume the message extract the value and use the data contained in it in addition to the key and value Fields a message also contains optional metadata such as a timestamp and a collection of headers the timestamp is populated by default but available for us to override the headers are useful for situations such as providing the type of serializer that should be used to deserialize the value however be careful when populating the metadata if data is critical to Downstream consumers
it might make sense to put it into the value rather than hiding it in the metadata for example if the timestamp of our event is important then perhaps it should be added to the value rather than populating the timestamp field now that we know what an event is and how it can be represented in Kafka the next step is to start producing and consuming those events inside of our applications if you aren't already on confluent developer head they're now using the link in the video description to acces
s the rest of this course and its Hands-On exercises foreign [Music]
Comments