VMSS Flex Spot Mix

Hey everyone, in this video I want to talk about the virtual machine scale set, flexible orchestration mode, spot mix capability. If I think of Azure as this huge hyperscalers, it has to have a huge amount of capacity just waiting for someone to go and spin up resources to consume it. Well that's really wasteful. And So what they do is they let customers use their spare capacity at a greatly reduced rate. However, it's on the condition that they can kick you off if they need it for regular pay a

s you go consumption. Now this is really good for me as a customer if I have resumable interruptible workloads. So hey, I want to get this job done. It's not a huge time crunch and I want it done as cheaply as possible so as a customer I can create virtual machines. And virtual machine scale sets that says use spot. And I can either say look I'll pay whatever it costs until I'm kicked off, or I only want to pay this amount of money and we see this in the portal. So if I jump over really quickly.

If I was creating a virtual machine, you can see I have the option to say hey. I want to use spot. And then I can say, well look, I want to keep using spot until you kick me off. I don't care if the price changes a little bit or I only want to pay U to a certain amount of money and what I can do is I can look historically. What reads am I currently in if I pick for example? Really matter, but if I was to just pick Dv 2. I could go and look at well, what is the typical pricing and it's a greatly

reduced rate for the region and regions close to it. Now this is a standard B2 which is burstable, which doesn't use spot, but here we go. I can go and say hey look. So right now Canada Central is much, much cheaper and it's also showing me what are the typical eviction rates. So E US is barely ever evicted. Remember evicted is hey a regular pay as you go, workload needs it kind of the central hey it's a bit higher. So I could go and make an assessment of what would make the most. Sense for me.

So that's the point of spot. I get it a greatly reduced rate, but I could be kicked off. And I've got a whole video on that. And then you have the idea of VMSS flexible orchestration. so we're used to the idea of VMSS uniform. VMSS uniform is I have a template which also has some configurations. I can have a minimum, a maximum number of instances. I can manually scale it. Or I could have autoscale, hey add instances if CPU is above this certain percentage. Or maybe a queue depth is whatever tha

t is flexible gave me the ability that I could add just regular VMS into it and I could still have a profile from which it could create virtual machines, IE that scaling capability. But the point is I could have a mix, I could have a mix of different VM SKUs, I could have a mix of spot. Non spot. I had a lot of flexibility, hence flexible orchestration mode. But it was still really useful because hey, within that flexible VM set I could have orchestrated patching and extension upgrades. I had fu

ll control availability zones and fault domains. I could still have that scaling and automatic scaling. If I did the optional profile I could have a bigger VMS up to 1000 instances that used the regular. Newest ARM models to create, so it was more successful and the VMS it would create from this profile were regular VMS, I could fully interact with them. So it was really, really powerful. So what's this video about? Was about spot mix, and exactly as the name suggests, I can now have a mix based

on that profile where automatically creates them of regular and spot. So in the past I could use spot, but they'd all be spot when they were created from that profile. Now the whole idea is, well look, maybe I want regular VMS for a certain amount because they can't get kicked off so they're more reliable, but then I'd like to. Have some extra capacity in my scale set if I can, but I totally get if need be. It's going to kick me off, but I'll take advantage of that cheaper resource while I can.

Now as part of this has a couple of parameters we configure when I want to use this spot mix, and actually if we jump back over to the portal for a second we can see them really quickly. So I'm going to go back and we'll look at the virtual machine scale set and we'll create a new one. And as usual, I just have to put in some of the basic options for creating itself central. I think none of the other stuff matters, except I have to make sure I pick flexible. For the orchestration mode. So that'

s the first thing I have to do, flexible to have spot mix and then I'm going to pick a size that supports the spot I'm going to turn on. This option to run with the spot discount. The ability to then set that mix is going to be exposed to us in a new tab. So we now have this tab spot and that's why I'm going to now be able to say. Well, firstly the eviction. So is it just evicting me based on our pay, whatever that variable cost is? The price varies depending on which region and what capacity st

rain that region is currently under. So it will vary. Sometimes spot will be cheaper when there's a huge amount of capacity, sometimes it would be a bit more expensive. So I might say, hey, I don't want it running if it's creeps above this amount of money. So I could do price or capacity and then I would enter the price or I might say, look, I'll pay the variation, only kick me off if you have to because you want it for a regular pay as you go. I'm going to ignore this eviction policy thing for

now, but it's super important. But now we have this option to scale with VMS and Spot VMS, and there's two configurations I'm making. The first is I can set a base number of regular inon spot, and then once we get past that number, what is the distribution? So I set a percentage of regular, IE uninterruptible. And then whatever the rest of the percentage is would be spot O. In my example I'm going to say, well, I'll have two bass. So it's always going to make sure there's at least two regular VM

S, then anything past two we'll set it to a 5050 mix. So if I put 50 regular. And it's doing 50 spot. If I said 25% regular, well now you can see it's doing 75% spot. But this only kicks in again once I get past. That base number so. Those are the really two important configurations when I think about this. So those are a key point so. If I was to jump back I guess for a second to the whiteboard, we could start drawing this out O if I think about for a second, OK. So make sure. We're using VMSS

flexible orchestration mode, that's our step one. And then we configure this idea of the mix policy. Now it's called a priority mix policy is the proper name if you look at the ARM template, but it's two configurations. So the first is that base regular priority count. I'm just going to call this regular number. Of base. So in my example, remember we're setting this to two. And then I put in. Well, what is the amount? If I go past two. What percentage of the new instances passed the base number

do I want to be regular? So if I write in 50 for the 50% that would obviously mean my spot. Would be 50. If I set for example the regular 25. Well then my spot would be 75. So you get the idea. So I set those things. That's pretty easy to follow. But remember the whole point, and it adds a wrinkle to our otherwise very pleasant day. Spot is that spare capacity so spot can get evicted. So if I think now for my spot configurations. We have this idea of well, what do I want to do? In an eviction po

licy. If that capacity is needed and they start kicking my percentage of VMS, maybe one of them, maybe all of them, whatever that might be, what do I want to do? And I can pick, I can say, well, Deallocate. Deallocate means. That VM still exists, it's just it's like I've done a stop deallocate on a regular VM, it's still exists as an object. But it's not running. I'm not paying compute charges on anymore, or I can say delete. Just delete your whole object now normally with VMSS when I scale in.

We delete because they're typically considered stateless. They're these tin soldiers. They're they're all the same when it's built off of the profile. So we just delete them. Remember why it's deallocated. I'd also still be paying for storage with spot. Remember I can't use ephemeral storage because how would I deallocate it once it's disappeared? Ephemeral says use the local storage on the host. Well, if I'm deallocating, I can't use ephemeral storage on the host because it doesn't exist on a h

ost anymore. So we'll still be paying for the storage associated with that. So this eviction policy is an important point and we see this. If we go back over. So when I'm starting to use spot, I have to configure this eviction policy. Now the default is stop deallocate and we're going to see it later on. I don't like that at all. I would recommend so I'm going to say this many times. I would recommend we do delete today. This will change in the future and I'll explain why as we go through the vi

deo, but I would pretty much always set that to delete. And in my example I'm going to use a 5050 mix just so it makes honestly the math and the following along a bit easier. OK, so we have our configuration. So let's walk through a pleasant day scenario of what would happen. Now. If I think remember about my VM scale set, it has a certain capacity, the number of SKUs of the VM that currently exist. And then we're going to have this new mix of some are going to be regular, some are going to be s

pot, and then we're going to have scale actions. Now when I talk about a scale action, this could be me manually changing the current number of instances. One through the portal, through an API or maybe I've configured autoscale. Hey, the CPU is above 70%, add two. Hey the CPU is below 30%, remove two. Hey a queue depth is above 200, add four. Hey, the queue depth has dropped below 10, remove 2. Whatever you want to do. It will work exactly the same way, it's just whether I'm manually doing it o

r I'm using rules configured in the VM scale set to actually go and do this for me. OK, so let's walk through an example. So I'm going to track four different values as we go through this. I'm going to think about, well, is there some scale action? Occurring, adding or removing, I'm going to think about what is the current capacity. I. How many VMS do we currently believe there are in this VM scale set? Now we're going to have a certain number of regular. Non spot and then we'll do spot in a dif

ferent color and then a certain number of spot. And let's say we started off just with two, so we start off with two. Now remember. If we look at our configuration we're using here, our regular base is 2O. Until we get past two, it will only use regular virtual machines. So we start off with two, which means all we're going to have is regular. We just have two. Regular virtual machines. Great. Now through some scale action we're adding four. This could be me manually doing something wrong please

, this could be me manually doing something. Or. Autoscale rules have triggered, so I'm going to add 4 O. What this now is going to say is, well, I'm past two. My mix is this one. This was the mix we're actually using in this example. So it should be a 5050 split. So if I'm adding four and I'm past my base, we should do two of each. It should do two regular and two spot. So now it's going to say OK. 1. Two, we had already add two more. Fantastic. And then go and create two spots. So my total cap

acity is 6. Perfect. That's the way it's going to work, so it's drawing scale out. And scale interactions. It will try and get as close to this mix percentage you specified as it can. It's never going to try and fix that balance any other time. So if it's just regular running, it's not going to say, oh, I'm out of balance. I'll delete some regulars, or I'll delete some spots and add some regulars or add some spots. It never rebalances. It only is going to get as close to this break percentage as

it can. This mix drawing scale out and scale in actions, that's the only time. There's never any other kind of rebalancing that is going on. And now let's say, OK, things are looking good, we're going to delete two things are quietened down. Will again. It wants to maintain that 5050 split once we get beyond that regular base two. So we can say why should remove one of each. So now it's going to say, OK, well we have that base. Two, we've removed one regular, and we're going to remove one spot.

So it's maintained the 5050 mix that we've requested, we've asked for that 5050 mix. So that, that's so great, that makes total sense, that's easy to follow. But now let's add in that wrinkle of this is spot. It can get kicked out. So what we now have to think about is, well, which one of those eviction policies did we select? Because depending on what we selected, it's going to do different things. So initially, let's say we've gone with the eviction policy of. We'll say. Deallocate. So we say

OK, this was configured as D allocate. And we'll start off with the same scenario. Actually, we'll go a bit simpler for right now. So here. As you know, we'll start from here. Let's say we have 4. So right now I have a capacity of four. So I'm here. So no scale action right now. I'll go 4. Capacity, and I've got these. I've got the base. And then I've got two extras like 5050 split. I'm in this scenario. An eviction happens. Hey, we need this capacity for something else. So there's an eviction

event. This. It's no longer running. But remember. Our eviction policy was to deallocate. If the eviction policy is to deallocate, it still exists as a VM object. It's still there. So it's not running. But VMSS is not considering the power state of the virtual machine O. What that means is this gets evicted. But this is still got a capacity of four. Nothing has actually happened to this. This is now just going to sit there evicted, and here's the challenge, and this is what we're going to actual

ly build on. Nothing is going to fix this deallocated virtual machine. To make this run again. I would have to manually start it now I could again. I could write something. I could delete it myself, but it's just going to stay. As the allocated. There's nothing built in today to virtual machine scale sets that is going to try and restart this thing. Now, there is functionality in uniform that has something called try and restore, and that if a VM a spot VM gets evicted, it tries to find a new cl

uster that it could restart it on that has a statistically higher probability of staying running, so it's going to have a better survivability. That functionality does not exist yet in VMSS Flex spot mix, which is why I'm going to talk about, well, I don't really like the allocate. Today it will change when try and restore shows up, but it's not there yet so this is going to just stay there. Unless I have something to try and start it, I write something, it's just going to stay in that evicted s

tate. How does it ever get resolved? The only way this would ever get resolved is if now there was a scale action. Let's say we said hey, we're going to remove 2. At that point, remember. It tries to get back to its balance, but it still counts as just regular as no intelligence is this actually allocated or not but because it needs to remove 2. And it's already, it's in balance. It removed one of each. So that goes and that goes. So now we have a capacity of two and it has removed that. DSS del

eted it because scaling always deletes this policy we're talking about here. Is only when it gets evicted. Because of hey regular workloads need that capacity back. So this is not saying what does it do during scale in actions, scaling is going to delete the thing. This is if it gets evicted, what do you want me to do? And so hey, it got evicted, we had it set to deallocate. And so it stayed there. So the only time that gets. Actually cleaned up, unless I'm manually doing something, is when it s

cales in and it gets removed. It's a little bit worse is the right word. Let's go to a different starting point. Let's say right now we had a capacity of eight. And so I've got. My 2 bass again and because we have a capacity of eight and it's managed to maintain the correct mix. That means I've got two extra regular and I have two spot. And once again, there was an eviction, but only one of them got evicted. So let's say this one got evicted. OK. Now there's a scale action. Now it says delete tw

o. Remember what I said? It doesn't consider the allocation state the power state of a virtual machine. And so it has to pick one of these to delete and one of these to delete. There is no intelligence. Delete one of these. I don't know. Let's say it delete it maybe based on age, so it deleted that one. And in our ideal world, it would delete this one. There's no guarantee it does that. It may be deletes. This one, and so I'm actually left. With the evicted 1. Because that's the one that got del

eted. And that's a key point about this. Keep this in mind as soon as I start getting ones evicted, there is no intelligence used. To which one I should? Delete in that scale in is. There were some that were evicted, deallocated, and some that aren't. It's not making a decision based on that allocation state. So it's just super super important you understand this is the way it's going to work and the only time these deallocated ones get fixed. Is. When it scales in. So there's no intelligence ar

ound that? And This is why today. I'm not a fan of this deallocate. My guidance would be honestly don't use this. So don't use until we have try and restore. Or maybe there's some built in automation that's native? Maybe it's not trying to steal it. Maybe it has something else that will go and retry. If it sees deallocated ones, but this today the deallocated, I don't see any benefit. I guess potentially maybe there would be something on the managed disk. That you care about as you don't want it

to get deleted, you want to be able to go and get to that disk to go and clean something up. But I don't see what they're buys you. Typically the whole point of these, they're built off the profile. There's nothing unique or special about them. I shouldn't care because if they were scaled in I would get that deleted anyway, so I don't see an attractive. Point for using the allocate today once we have triangle store or something else and it would then restart it and re put it on a better cluster

. Fantastic. Totally get the idea of deallocate today. I think it leaves me in a very weird state and there's too many things I would have to go and clean up to actually go and do something. Now. Let's take the other one. Let's say it does delete. Let's say I set that policy to delete instead. It's much, much cleaner. So we'll start with exactly the same scenario we had here. So now if I had my 8. So I have my two base. I have my 2. Regular ones, because it's a 5050 mix, I have my two spot. And

now we're doing exactly the same thing again. OK. It gets it and I say both of them are evicted. It doesn't matter if one was evicted or two was evicted, but let's say these both get evicted. Ohh no they were evicted. But because my. Action is set to delete. When these get evicted, what will actually happen is the VM scale set says, oh, they've been deleted, they've been evicted. I'm actually going to do. It's just delete them. They just deleted, so it just deletes them both. Because they're del

eted. They no longer count in the scale capacity, they don't exist. So it's actually now dropped the capacity of the virtual machine scale set to six. Now my mix is completely off. Remember my desired mix is 5050. It's 100% regular. But it doesn't rebalance. There's no rebalancing until a scale out or a scale in. But. Now there is a scale action. Let's say now we say add 2. So it says OK, well I've got my existing my base count of two. I've already got 2 extras. I'm currently at 100% spot mix. M

y desire is to get to a 5050 O. What should I do? Well, the best thing I could do at this point is you know what, I'll add two spots. So it's adding these two. What has it done? Well, it's now got me a capacity of eight. But it's got me back to my desired mix. I now have this 5050 and that's the key point. It's going to get me to rebalancing as much as it can. If it let's say for example there were instances and it only evicted one of them and it was deleted and there's only one spot mix and let

's say I then scaled in scaled in action of 1. Well, it would have deleted one of the regulars. To get back as close as it could to that 5050 mix. So it's always going to try and get as close to that mix percentage I've set when it's doing a scale out, adding stuff or scaling, deleting stuff so it will try and create. Regular or spot that would get it as close to that 5050 as possible, so it's always going to try and get me back as much as I can. And This is why I like delete, because delete if

it does get evicted, it reduces the scale of the VM scale set and which means it's actions are going to do the right thing to get back to that priority mix. If I've got autoscale rules or how it's seen, the capacity has dropped, so maybe there's some trigger would make it now go and add an ohh it would go and create some again. So that's going to get me to that good space. Now there is another scenario with spot, hopefully that will make sense. So basically the guideline is today. Use this. I'll

add a note or thank in the description. If they add some. Automation that's native or they add trim in store because then I think deallocate will be really nice because if it does get evicted it will spring back to life on a better cluster with a better survivability. But it doesn't have that today, which means it just sits there. Yeah, I've fallen and I can't get up, but nothing's helping it up. I'm pushing the button. It's like, you know, on the floor. So I want to use it. But delete absolute

ly. There was another scenario though. Imagine we are doing a scaling action. But it can't create them. There's currently no spot capacity available. Now this because it's not being evicted. It just can't create. This policy doesn't have any effect. This is not used in this scenario. So let's say I start out with just my. 2. And this is a challenge it seemed to be aware of. Because I've got 2. And let's say for example I am using autoscale. Now, what if it can't create a spot? What if it's tryin

g to add 2? So I've got my 2 bass. It adds another one. And you should do one of each. So it tries to create one, but it fails. It can't allocate it. It will still create it, but it's deallocated. Now it's not being evicted, it just can't start it. So it's not like here, it will go and delete it. So in this point it cannot allocate. Now this is not a common occurrence, you saw some of those eviction policies. And of course when it's creating it's got an even better chance of being successful bec

ause it will go and find a cluster. Where it could create the thing. So this is not a common scenario, I'm just pointing it out to stress a point was the question comes U what if it can't allocate a spot? Would it create a regular instead? And the answer is no. It would just create it deallocated. Now the reason it doesn't create it is well, there's two things about this. The first is remember it doesn't consider the powered on state. If it exists, it counts towards the scale. So even though it

couldn't allocate it, this is still four. It thinks it's done its job. But the other thing is I've configured a mix. I've configured hey, I only want to have fifty as regular. The other fifty I'm willing to have that interruption. I don't want to pay for regular. So if it did standard instead, it's breaking that kind of agreement it's had with you as the user that will only do 50% as regular. If you just went and did a novel regular, that's costing me a lot more money, so it doesn't do that. But

this is just going to stay here. And here's thing I want you to consider. If I'm using autoscale rules, so there's trigger was actually caused by. For example, hey CPU is greater than 70%, so go and add 2 to get that CPU down. So BMSS thinks it's added two, but it's only added half of the capacity that it thinks it's done. So it's not as been as effective, it's not done as much as it thought it was doing. And so this rule, remember will refire. This rule will after some cooldown time says hey m

y CPU still high, add another two. Let's continue taking this horrible, horrible scenario. Remember, as far as it's concerned. It's at the correct spot mix. It doesn't consider the powered on state. I have one of each, I should add another one of each. So I've had another standard. And in a horrible, horrible day, this failed as well. But now it's capacities at 6. What it's essentially done is add 2 standards. Instead of 1 standard. And one spot based on my desire. It just delayed it a bit becau

se it had to get retriggered by the rule. So it's a little bit an interesting combination of things. Again, this is so unlikely. This is very, very unlikely it's happened, but I just wanted to point it out. That again, remember these would only get fixed. At a scale in unless I manually went and deleted them for example. So those will now stay there until I scale in. And remember it doesn't consider that powered on state. So if I just reduced by two, it would delete one regular and one of these.

I'd have to scale all the way back into two to get these to clean up automatically by VMSS so in that allocation failure scenario. Which again this is not likely. So I guess I should really kind of stress this point a little bit. This is very. Not common because when it's actually creating the spot it has a free range to come and look at what clusters where it can put things. I'm taking this little bit of a far out scenario, but I wanted to explain the point that does it use a regular? Instead

of a spot if it can't allocate, no, it does not. But if I'm using especially autoscale, that's triggering if it can't create and the half in this 5050 split scenario. Does create regular with. It doesn't do what it needs to do to address whatever triggered the scale the next time it's allowed to reevaluate that cooldown. As past. It's just going to repeat it. So in this scenario it has essentially what I wanted was one regular one spot. What I'll end U with is 2 regular running and two deallocat

ed spots. So in a way it has used the standard instead of the spot. Again, super unlikely, but I think it's an important thing to understand. That that could happen, that those things could occur. So it's just a consideration. Understand it's there. It's not saying you're commonly going to have to worry about unlike. This deallocate is super common this whatever that eviction rate is 5 percent, 15%. And then you are just sitting there with these deallocated ones, giving you no use. So again, do

not use deallocate. Until they have some native automation, some orchestration or try and restore, in which case, fantastic. It's going to spring back to life on a better cluster. It's going to have a better survivability. Until then, use delete. It's going to be much, much cleaner if it gets evicted. It's removed from the scale set, so it's not counting for capacity anymore, so future scale actions will go and get you back to that desired mix you actually want to do. So that was it. That's what

spot mix does, exactly as the name suggests. Hey, I have a number of bass. Anything beyond that is going to try and get me to my mix of. Regular and spot, whatever that ratio may be, and again, it only does that during scale out and scale in actions. So that was it. There's always. I hope this is useful. I think this is a fantastic capability. I think having that ability to have the mix is really useful. Hey, I need this guaranteed reliable set of capacity. But hey, if I can use this really che

ap stuff, that may give me some benefit. If it's affected, it's evicted. I've still got my corset to make sure it can do the core job it has to do. Maybe I can get some bonus bump in the stuff it's doing. Over here if that workload can handle that interruption. Till the next video video. Take care.