Azure IoT Hub integration with Azure Event Grid

>> Did you know that you can send device telemetry to Event Grid thanks to IoT Hub? Well, that's a new capability that Ashita from the Azure IoT PM team is coming on The IoT Show to tell us everything about. [MUSIC] >> Hi everyone. Thanks watching The IoT Show. I'm Oliver, your host, and today we have Ashita back on The IoT Show to talk about enhancements to the Event Grid Integration of IoT Hub. That's a lot of words. >> Yes. >> Thanks Ashita for coming to the show. >> Thanks for having me. >>

So real quick, once again can you introduce yourself and your team for those who don't know you. >> Sure. I'm Ashita and I'm a Senior PM in the IoT Hub team, and I focus on data integration type features that help our customers flow their data from IoT Hub to different Azure services or their own business applications. >> So Azure IoT Hub is becoming a very powerful device gateway that not only allows to ingest data from these devices, but also last to be smart in the way you're going to actuall

y route these messages to various endpoints, and services, and the rest of your application basically, right? >> Yeah. >> Until now we had an integration with Event Grid which is the Azure pub-sub infrastructure that was just about some of the creation and deletion of the devices in IoT Hub device registry. But now you can actually start sending to Event Grid telemetry messages. >> Yeah. >> So tell us a bit more about that unless I spoiled the whole thing. >> No, that's perfect. So, yes. So as y

ou mentioned, we already have an out of the box capable called message routing that allows you to automatically configure routes, and send data to different endpoints. We support only four different services as custom endpoints in IoT Hub. One being Event Hubs, Service Bus Queue, Service Bus Topic and storage. >> Okay. >> But Event Grid opens up that endpoint lists to many many other server-less application type services, and also third party applications. So I can send data to Azure Functions,

Azure Logic Apps, Microsoft Flow, or even my third party application through a Webhook. So that's a big benefit of using Event Grid. >> So Event Grid actually was mentioned as a pub-sub system. So in principle, you will then have IoT as a publisher, right? >> Yes. >> So you're a publisher on that infrastructure and any other service that actually can be a consumer of these topics on that pub-sub system can actually receive that notification. >> That's right. Exactly right. So Event Grid is actua

lly a universal back-plane that is connecting Azure services with other Azure services, and as more and more services onboard as publishers and subscribers, it's really easy to push events out to other things and trigger workflows. >> So I have a trivial question. Why do I need to go through IoT Hub? Couldn't a device directly post a message on Event Grid to trigger something? >> IoT Hub gives you the benefit of having per device identity and security, which is really important in IoT scenarios

usually. So that's a big benefit of going through IoT Hub. >> Makes sense. >> Yeah. >> I think there's another aspect of it because now that we have that Event Grid integration, why will we need to eventually still send and wrap message to Event Hub, or topics, or what's the difference and when would you use which one? >> Yeah. So routing allows you to give ordered messages out. So it guarantees ordered delivery where Event Grid doesn't, but Event Grid gives you the option of expanding that to m

any endpoints and many types of endpoints. But routing you can also use because it's free, Event Grid is going to charge you to ingress the data to Event Grid, and then egress, even though it's very smallest at $0.60 for a million events. >> One million. Okay. Yeah. >> Yeah. But routing also has very rich queries. So you can actually apply querying on device twin as well as the body and then routes default. >> Okay. >> So for many customers, 100 routes are enough, but sometimes they needed to go

to more. >> We're talking about the scale, because like eventually you have a big IoT application with like millions of devices. What kind of scale are we talking about in terms what Event Grid can ingest? >> Even Grid can Ingest about 10 million events a second, so it can handle up to scale. >> So we're good? >> Yeah. >> Awesome. Let's look a bit more into how it works, and show us a demo. >> So as you mentioned, we actually have four Device Life-cycle events today which are available. Like de

vice created, device deleted, device connected, and disconnected that can trigger workflows for customers in near-real-time. So we've added a new events device telemetry which is basically the device data that comes from the devices, and it's going to Event Grid. >> Okay. >> Here's what the events schema looks like, and I would just like to go through it real quickly. That every event that comes from IoT Hub to Event Grid will be put in this envelope from Event Grid, and what that entails is an

ID with is a unique identifier for that event, a topic which is which resource gave this event, and in this case it'll be our IoT hub, subject is the device ID that produced this event, and then the event type and time, which is what time the event was generated. Data is where IoT Hub populates what needs to be in this event, and here we have basically the IoT Hub message format, and the body that is coming from my device. The properties are obligation properties, and then system properties. >>

So I can put, basically if I have, a bunch of data that I put into body eventually it's binary whatever, it will still flow through Event Grid [inaudible]? >> Yes. It will still slow, but it'll be in the Base64 format. >> Yeah. Okay. >> If this content type and encoding are set to JSON and UTF-8, it's more powerful because you can then actually filter on the content of this in your subscription. Yeah. >> Got it. Okay. Makes sense. >> So in order to demo this, I'll do couple of things. I created

a logic app to post a message on a team's channel every time an event occurs, and then I'll go and create an event subscription, and then we'll test it with a device simulated. So let's first look at the logic app. So here I have a workflow where anytime each of the request is received, I am going to take an action of posing on my team's channel, and in the request I actually was able to put this event schema into there so that logic app was able to parse it, and I use that to create a message.

>> Okay. >> All I have to do is basically copy this HTTP link, and I'm going to use Webhook as my endpoints. >> Okay. >> So in IoT Hub, we have this Event Grid blade. This is a universal blade that any service publisher for Event Grid will have the same look and feel. >> It's not specific to IoT Hub. It's just, I'm the publisher on Event Grid so here is the look and feel in the data I'm going to put in there. >> Right. >> Okay. >> So I'm just going to add a name to this subscription, and you can

see here that we have four events pre-selected which is all of our device events. So let's say I want to add telemetry to this, and my endpoint is of type Webhook, and all I have to do to create this is paste my HTTP link here, and that's all. So it's that easy. Now let's test this. So here's my Teams channel, and what I'm going to do is I'm hoping to see events here. So let's go and look at a device simulation. So here is my device I'm sending, weather, temperature, location, and I've set my c

ontent type encoding to JSON UTF-8. >> Nice. >> I'm just going to play this, and it's going to send me telemetry every five seconds. >> Okay. Sent. >> I am hoping to see data here right away. Here we go. >> Yes. Pretty straight forward and fast. >> Pretty fast. So it's near real-time. >> Nice. Actually is there any, you say near real time, do we have SLAs in terms of to Event Grid messages ingestion delivery? >> It does guarantee delivery, and we'd retry 24 hours. >> Okay. >> So IoT Hub has a re

try policy of one hour, so it'll try to send the message to Event Grid with retries in an hour. But then Event Grid will try to send it to its subscriber in 20 products >> Okay. When you are developing an application into one in the Cloud anyways, you can actually instrument the application to monitor the errors that the services will send, and so if a message was never sent, and after retry it, you should actually monitor that there's been an error in the message sent and then react in your own

application? >> Absolutely. We have rich monitoring metrics for both these things. >> Awesome. >> So now just to explain what actually happens behind the scenes. Whenever we create a new device telemetry subscription, IoT Hub is actually going to create a route to Event Grid. >> Okay. >> Now, note that you may have multiple subscriptions, but there will be only one route because for IoT Hub, Event Grid is one single endpoint. >> Okay. >> In this route, what users can do is actually filter data

go into Event Grid. As I mentioned a vendor charges when you ingress data to Event Grid. If customers do not need to send all the telemetry all the time, you can easily go in here and just change the query. Note that nothing else is editable, so the data source can only be telemetry messages, and that's when the route is created for telemetry. So I can go in here and actually I can say, $body.Weather.Temperature. Sure is greater than let's say 25. >> Okay. >> So my device data is between the ran

ge 20-30. >> Okay. >> Now, I'm expecting that every time the data is above 25, only then I will get the message. >> Got it. Anything below will not be sent? >> Right. >> Nice. >> So let's say I go to 26 and then 24, let's give it some time to see the route. >> We should only see the above 26. >> Above 26, right. So you notice 24, 21 will not show up, and then 29 will show up. >> Awesome. >> Right. >> Love it. >> Now, another thing in Event Grid is that we have the capability to filter in the sub

scription, because as I said, you may have many subscriptions, but you can query on one route. >> Okay. So we can actually go in the Event Grid subscription, and this is a very advanced filter where you can filter on the body of the message as well. So all I have to do here is add a new filter. My key value is going to be my data. >> It will be we fair to say that once you've adopted the Event Grid infrastructure, basically you would recommend doing the filtering there, right? >> Yes. >> Okay. >

> The only advantage of routes is you can filter on device twin as well. >> Okay. Yeah. So let's say I want to do this now at 27 for this particular subscription. >> You can actually also, as you're doing, piling on the various, there's the route and then there's a filter? >> Yes. Yes. >> Okay. >> Yeah. The concept is very similar. You basically have a data source and an endpoint where it goes in a filter. So it's pretty similar. So this is saved, and I'm expecting to see data only with when I h

ave 27 degrees. So I go to 21 and 23. >> These ones will not show. >> That's right. >> Then if you have changed so you're going to get something in between 25 and 27 that will not show in here because of that second filter you just have. >> That is correct. Yeah. >> Okay. >> Yeah. So I think that just showed up, 25 did not show up between the Intune. Yeah. So this is very useful when you are pools processing data from many, many devices. >> Yeah. >> You have micro services that need to only get

to particular set of data, and if you're running different functions every subscription can have a different filter. >> I love it. >> Yeah. >> So if people want to learn more they can go to that aka link that you have up there, which is aka.ms/ iotshow/telemetryoneventgrid, one word. Awesome. Ashita, thanks a lot for coming on the show again. I really look forward to your next appearance to talk about IoT Hub messaging, and all the goodness in there. >> Thank you. >> Thanks all for watching The

IoT Show. Don't forget to subscribe, and I'll see you soon. [MUSIC]