this seems safe okay a few weeks ago I'm browsing Facebook Marketplace and I stumble across this a once top of the line NetApp storage appliance that would have cost around a hundred thousand dollars new for five grand and it gets even better we low bulb them offering just a thousand dollars and they said yes that's how desperate they were to get rid of this thing which raises some interesting questions like why does it even work I didn't follow my own advice and actually power it on before we c
arted it away so for all I know it's completely dead and even if it's not dead is it so slow by modern standards that it really is only worth one percent of what they originally paid for it there's some way that we could D proprietaryify this thing and roll it as commodity storage Hardware we're going to have the answers to those questions and more after this message from our sponsor unbounce engage your audience with beautiful high performing marketing campaigns in just a few minutes with unbou
nce's AI powered landing page builder on bounce on over to the link below and get up to 35 off for those who missed part one of our adventure let's take a quick look at what we've got here a NetApp storage Appliance is not that dissimilar to the kind of Nas that you would build for yourself just everything about it is Enterprise grade instead of having a single box that contains everything though it's split out so we've got this box here that contains our brain or our controller for the applianc
e and then this particular configuration has four 24 Bay disk shelves which are as the name implies shelves that hold disks these ones in particular have 600 gigabyte 15 000 RPM SAS drives in them and then just power supplies at the back in order to power them but here's the thing a hard drive is not very useful without a data connection but remember how I said these were SAS you can think of SAS kind of like the Enterprise Big Brother of SATA and each of these drives is connected to a back plan
e inside our disc shelf that is then connected to our controller through an external cable those of you who have been paying very close attention will know that this is almost exactly like our mother Vault server keyword being almost unlike the mother vault which only has a single controller because it doesn't host Mission critical data this is designed for what's called high availability which means that no matter what in this configuration fails there should be something ready to pick up the s
lack immediately so that there's no interruption to your business and it starts with of course the controller box which contains not one but actually two separate servers Each of which needs to be connected to every single one of these j-bots see like that then there's the j-bods themselves which contain redundant power supplies so I could literally rip this out mid operation and it would keep working contain redundant controllers so I could have a controller failure and every Drive in there wou
ld still work and even have two separate cables connected to every Drive with all that said the reason I got this at E-Waste pricing is because it kinda is and not just because of its age you see NetApp locks down their systems so that you can only use them with drives that have customized NetApp firmware and I'm sure they'll tell you that this is for your own benefit in some way but the real reason they do this is to ensure that you won't be able to take advantage of the ever falling pricing of
commodity hard drives and you will have to pay the NetApp tax for ever with that said though if we abandon the NetApp controller which frankly isn't very performant or power efficient these days anyway we should be able to use these disk Shelf jbods with practically any server or even a desktop computer which makes these things pretty valuable in the home lab Community as a cheap way to run a bunch of hard drives all we need is some adapter cables a controller server and an HBA to plug into but
we're getting ahead of ourselves a little bit you see neither Jake nor I has ever gotten a chance to play with NetApp anything and despite paying a thousand dollars for the entire get up we haven't powered it on so why don't we start by taking a look at how it's supposed to run the first thing we'll need to do is wire this up properly uh what I did earlier was just for illustrative purposes these both need to come out so we'll start by taking the leftmost port on each of our servers I apologize
earlier I referred to these as controllers and they are in a sense because they're the controller for the entire storage Appliance but these are also SAS controllers so I'm just going to call these servers and these controllers moving forward I plug each of these into the two controllers on the top jbod just like that then one by one top to top bottom to bottom I daisy chain in each of the additional controllers then once we reach the bottom jbod we actually Loop that one back up around to the
servers and the reason for this is that if there's a failure at any point in the chain it can just go oh well I'll just come at it that way there is four power supplies per disc shelf you technically only need to plug in the top and the bottom so that both controllers are powered wow that's loud I hope that spins down later yeah she goes no power buttons on this oh wait is this one bad maybe it's just not plugged in oh wait hold on is it not plugged yeah those are the same lights that are bad on
the yeah yeah ones that are unplugged so okay does it push it in maybe oh no it's in there try the other side yeah I'll try the other side oh did you just not turn the switch on oh you just didn't turn the switch oh I missed the switch on that one there you go high defense it took you a minute to see that switch too it's right under the thing well I mean in my defense you were flipping all the other switches it'd be kind of weird to miss them I can't really serial into both of these at the same
time so I'm hoping one is enough you think the top one is the primary good thing all right let's try that the only link lights that are not illuminated are the ones that are coming straight from the servers if it's not booting up that makes sense I also don't know does it need boot drives did they take boot drives out in theory we should have cereal even at the bios so let's try that I guess okay it looks like it's doing things nvrd one I'm assuming that's like the non-volatile dim interface e5
a e5b does not exist were all those management ports plugged in it wasn't in their server room anymore so they wouldn't be nothing plugged in here I really don't know if it needs boot drives oh hey look here's all the instruction manuals they still have them on their website oh my god wow baby's first storage cluster data for seven mode 10 gig ethernet ports so that makes sense we have those plugged in between we have our SAS ports that's right connect gig ports blah blah blah to Gig Data Networ
k switches okay all power cords must be used for all units yeah whatever what's the point of even having a manual okay so the whole thing has to be Daisy chained every single jbot so the farthest drives from the actual SAS controller it's just because if one connection dies or like something goes wrong the connection can come back the other way okay well we can fix that uh let's flip the power switch okay now we get to do it all over again with ethernet ah yes the locked wrench Park transport to
the first ones ah good quite I screwed up one of mine what uh it goes top to bottom then bottom to top so I just gotta do a quick uh switchy boo here okay those are Blinky that's good oh wait what how did I end up oh God we're still not getting a link light back to the server well it's not booted so ah oh the link lights are up it's doing stuff yeah something is happening now deliver monitor take over disabled blah blah blah it's doing things but these controller nodes each have a two Port 10 g
ig card in there we didn't account for that in our like money oh yeah those are worth like 50 100 bucks each yeah I think we did pretty well here in spite of the fact that they ended up telling us after the fact that the five thousand dollar listing was a typo it had intended 500 dollars system was down for 5.3 million seconds what the hell is that not saying that was the last time it was on well no I oh it's Blinky it's got internet even oh my God what is the login install and run system setup
software oh register and get your license Keys maybe you can download this part of it we could maybe be able to download the system setup one but anything further than that I doubt download system set of 3.1 or later 2015 was the last time this was updated oh no we wanted to check the power consumption because 60 terabytes or whatever what does it work out to Yes it really it's only 60 terabytes yeah and you don't actually get all of it some of the capacity I think minimum three discs have to be
used for each node to host their OS so the reason it wouldn't turn on after talking to lead eater on the Forum is one of our form admins shout out red eater the OS is stored on the array if you move the disc shelves around and change the cables and move the discs around yeah okay fine 45 terabytes whatever the capacity ends up being that's not a bad thing you know I wouldn't mind you know tens of terabytes of storage but at what cost these days you could get literally two hard drives and that w
ould be a total of what like 15 20 watts yeah the cost of running this compared to buying two or three new hard drives this takes a while to turn off yeah it is off sort of it shuts off the OS and it goes back to the loader which is like the bootloader menu and I guess that's as off as it goes and then you just click click the button I guess so yesterday trying to get it set back up cleared the config and initialized all the discs which is just one command but that means it has to zero every sin
gle disk and since this is on an old operating system it zeroed like sequentially and did that overnight sure oh here's the other thing if you update it from the older version to the newer version faster row doesn't work you have to fresh install in theory this one can be updated to a version that has pass zero but we can't download the software because we don't have a license I emailed them and said hey we bought this from X company but they said oh you didn't purchase it from NetApp so we can'
t give you access so it literally is e-waste basically all I'm saying is it should be illegal to sell Hardware that requires software that you need an account to access all right let's go three amps a 208 volt whoa holy crap it's drawing 15 1600 Watts right now eight freaking amps for one year it will cost us two thousand dollars in electricity here where electricity is like 13 cents a kilowatt Canadian which is like 10 cents a kilowatt U.S so six grand a year in California 10 grand a year in Eu
rope oh look at how like archaic that PCB is on the back well yeah it kind of looks like carbon fiber oh man look at that that is just like industrial AF we're not going to destroy all these drives if somebody in the comments you want them for your NetApp and you live locally and you want to just come pick them up and take them out of all of these trays maybe you can have them well NetApp is on now do you want to take a quick look at it I do this is quite uh the journey to get working so after I
installed and zeroed all the drives I booted up the second node which was still crashing and had to do the same thing luckily it only zero to three drives and it took like an hour we got old school cool bottom and top once you set up the cluster on the first node after it's reinstalled itself you can then reinstall the second node join that to the cluster but wait it's not high availability yet you have to go in and like type a command that allows it to be high availability like turn High avail
ability on otherwise it's just two separate notes So in theory now both nodes can perform takeover on each other it used to be that you needed a base license to even use the software at all they've changed that so you can install it but to do basically anything sifs like SMB NFS iSCSI you need a feature license and NetApp won't give it to me because we're not the owner yeah basically right now this is just an expensive storage device that you can't really put anything on as far as I can tell bec
ause here like I made a aggregate which is like a pool and then I made an SV honestly I was about right we only have 45 terabytes of actual addressable space which is a storage virtual machine you go to edit and I want to turn on sifts which is SMB okay and it's just grayed out you can see the cluster based license is deprecated but all of these other things you want to use any of these features you need a license because we didn't buy it from them or a reseller oh hey look a hard drive look how
small the platter is so the reason for that is that a smaller platter actually gives you faster seek times because the arm doesn't have to move as far to find the correct track that's one of the reasons that these high performance SAS drives in appliances like this used smaller disc platters the bigger the platter gets the more actual area that is after all of that work to get this thing to work it still doesn't really work let's do what we intended to do then and try and repurpose this Hardwar
e okay cool I get to run that fun really long command again to turn it off classic Jake see you later and the like condescending attitude of the people who have told me how stupid I am over the years for not buying one of these yeah nothing we have is that mission critical especially when the cost is so high that you could literally hire a person to come in and deal with it for you after your hand whoa I forgot about the handles hey hey how you doing yeah this is the most junk part of the whole
thing I think easily worth the thousand bucks right here even if we just give them away at the Christmas party and then what did you have in mind for a replacement what now we're gonna put old wanak in there oh my God what is it like dual high performance 32 course or is that one dual 64 7702 64 cores okay I don't know if this is ever what liquid intended this thing to be but um no I think they really didn't intend for hard drives the first thing you're going to need to make a disc shelf like th
is work with a different computer is an HBA or host bus adapter this is effectively just a storage controller on a pcie cart like what would be on your motherboard just separate this one is a 9305 16 I the eye is standing for internal you can see that these are internal connectors I see the problem here this is not what you should buy you should buy one that ends in e for external which has the proper external mini SAS HD connectors we just had a bunch of these laying around and startech makes t
hese internal to external adapters so this is really janky it's not that bad okay it's it's not that great Enterprise grade Hardware that literally looks like someone hand soldered the wires onto the PCB I mean why have a housing and a connector when you can just do this and put hot glue over it you'll notice this card has four connections and we have two of these adapter things that's because we have four disc shelves you could Cascade them and have one cable that goes to all of them I don't re
ally have the right cable actually no we could reuse them could you reuse it it's slower to do that though yes we would rather have more bandwidth this means we have a full connection to each of them I maintain it won't matter for the use case for the use case where you're going to be more limited by your random i o re-silver replacing a disc okay In fairness to you this ended up being less stupid than I expected yeah it's really not that bad get kick hey hey hey hey hey hey kick just stop kicki
ng stop the one that we got uses iom 3 controller modules those are SAS three gigabit luckily you can buy iom 6 modules which are SAS six gigabit which doubles our throughput uh how much money did we waste on those 20 bucks oh they're 20 bucks each wow that's cheap and we technically only need one as far as I'm aware the good news is that the upgrade process is as simple as oop so I'll pull out old ones you put in new ones yeah upgraded beautiful now we have double the bandwidth double the throu
ghput these are qsfp which is what the NetApp uses too many SAS HD which is like the modern external SAS cable I don't know if anything other than these use qsfp because this is usually a network cable connector like for 40 gig or 100 Gig networking yeah I mean in theory it's anything that needs a lot of pins handle feels like it's not strong enough it really does it's flexing so hard not as hard as I'm flexing man hold on the plastic thing who cares about the plastic power cable no I just it mi
ght have interfered with that oh it would have been fine it might not have been things like 100 pounds Linus actually dropped one of these oh my God a foot off the ground onto pavement and all the drives still work seemingly wow wiggle while you work look at how mint that is Buddy I mean from this side it's mint I really hope this just works I don't think it's gonna just work really why wait for Jake when I can just turn it on myself food failed booting from Debbie just give it a second oh okay
I just didn't change the boot order oh no my 100 useless capacity High wattage drives aren't all detected oh no oh I think we only got one shelf yeah we have one okay how do we know which one it is though oh I have no idea do an identify thing on the thing and I'll tell you what light flashes show me an Access let's go hey it's the top one are they all on yep all good I want to point out that as much as we're mocking the power efficiency of the original setup only a small part of the problem was
the server itself and now that we're running a new server that's not a problem at all also the power efficiency of these disc shelves is only a problem if the capacity is super low if we were to put 22 terabyte drives in this thing obviously these power supplies aren't quite as efficient as something modern you know 80 plus titanium would be but your energy usage per capacity would actually be much more reasonable and could even be worth the cost savings if you were trying to set up a very low
cost bulk storage solution for a small business honestly if something like this had existed back when we started just cheap j-bod disc shelves like this I would have totally gone this route they existed they just weren't low cost hey some of these drives are different capacity 556.29 and 555.63 that actually makes sense because some of them are the two and a half inch replacement drives and they would be ever so slightly different even though they're both 600 gigabytes failed to wipe disk invali
d argument I mean that's the whole point of their locked off firmware BS but why would they care if you put these discs into something else because see you later buddy knock to addition screwdriver coming soon you can sign up for an in-stock notification on lttstar.com two colorways this is interesting the two and a half inch drives have a little adapter magic oh it's just a height adapter they've got a screw in from the bottom I have a thing for that hey oh interesting this is an interposer we
don't need these a benefit of using an interposer like this is that you can use sas-ish signaling up until the interposer and then it converts to say to right at the drive and this has the added benefit of allowing both controllers to see the drive which is super cool but the thing is most SAS controllers can read SATA drives it doesn't work the other way around you cannot run a SAS drive on a SATA controller well without an interposer at least you cannot but the point is we don't need an interp
oser to read these drives necessarily it just is a little better one thing that's a little bit confusing are there two separate mounting hole there are there's two separate mounting holes on these sleds because they are interposer ready wow that's pretty cool we don't have that many spare drives right now but the drives that we're going to be using 22 terabytes it's still only detecting 25 discs McCall Wendell uh hello hello is this level one tech support well technically he's level two so we go
t the death those disc shelves set up they're all into the same HBA and it only seems to detect one of them es yeah the one it detected is four I have a 9305 16i HBA that's what they're plugged into [Music] the IDS maybe it's just mad because the IDS aren't sequential I think you are hitting the button on the front Wendell suggested that part of our problem might be these disc shelves complaining about having 15 000 RPM drives in them with only two of the power supplies plugged in because I see
so much more power it'll force you to use staggered startup and you gotta have three out of the four power supplies for 15K so we are trying to plug in a third power supply for each of our shelves to see if this resolves this just weird issue where the top one is detecting drives but the other three are not clear SCS util show it still just shows the one it's only detecting seven drives all the lights are up on the front there was some kernel messaging happening when we were unplugging and plugg
ing in enclosures which makes me think that things are happening but it really just wasn't detecting anything like you run SES util and it still would only show the one enclosure after talking to Wendell for a while it's become pretty clear that regardless of how we had this thing hooked up directly to each disc shelf or into one and cascaded down to the others this should just work there isn't really any config that needs to happen to make this work and it turns out it should have been Wendell'
s setup at home is running Rocky Linux which is different than FreeBSD which is what trunass core is based on on the off chance that it might work we decided to upgrade to trunass scale which is actually based on Linux and as soon as it turned on I immediately saw all the drives start populating so I guess it's an issue with true Nas core which is a little bit ironic because I specifically chose that thinking that it would be more reliable and better suited for this and it turns out it is not th
e weird thing is they show as disc size zero I think it's because of the weird sector size these drives with a special NetApp firmware use a 520 byte sector size which is not standard for most computers but it does turn out you can change them I'm running it on this one right now it's probably been going for like 10 minutes already and it's at 45 or something like that you can do multiple at a time so it's not going to take like a week to do all of these drives but I'm also not gonna do all thes
e drives they're really not that useful for us I just wanted to demo it on one so we'll see when that's finished and see if it works but in the meantime why don't we throw in these 22 terabyte drives I grabbed eight of them and set them up on the trays with the interposer so we can do two per disc shelf the other thing that's important about this while you can use SATA drives with these disc shelves when you have SATA drives and SAS drives on the same backplane it's really not recommended becaus
e the voltage level is different between SAS and SATA so using interposers in a mix setup like this is very recommended I'm not even going to try and put just to say to dry in here because I don't want to damage anything oh luckily I have this like server shelf here when I set up my server at home I really wanted to get one of these disc shelves but they're like particularly in Canada it's like oh 300 bucks on eBay plus 700 shipping like what they're blinking that's a good sign let's go see if i
t works it takes a sec for them to populate but I can already see the number it's increasing around 87.88 there's another one popping up this is still going 45 eight percent hey two three four five six seven eight hey look at that plus we have a direct connection to each jbod which means we have like not 48 that would be 12. we have 24 gigabit to each disc shelf which is two and a half gigabytes a second something like that which is more than two drives we'll do hey look it works there we go oka
y manage devices look at that all that's left to do is create a data set and then a share so we can access it I'm going to change it also to Z standard compression we have a lot of CPU in This Server 128 epic cores and that's just gonna serve to make our discs go faster so there's really not really a lot of downside I heard we switched to True now's scale yup how often have you started on core lately for the better reliability and more maturity and then ended up on scale I don't know what it is
core very much seems to be built for like IX systems which is a company that makes true Nas yeah the hardware that they sell these days and scale you know it's Linux it kind of just works with everything and sure enough as soon as we booted it up it was like wow look at all these discs really that easy right now I'm just plugging in that test Bench way the hell over there to 10 gig so we can like do a file transfer I called it the pool is called sort of slow this is Wi-Fi oh and then the data se
t is sort of fast we're directly connected to the server over there via 200 foot cat68 cable now to this workstation which has a 10 gig Nick in it we have two 22 terabyte hard drives in each it should be able to hit about a gigabyte so it should be fine okay wow that's reading from it immediately 1.1 gigabytes a second it just saturated a 10 gigabit link yeah yeah [Music] okay 600 megabytes a second I mean it did like five gigs before it slowed down that's probably just a little bit of ram cachi
ng there is nothing to be too concerned about this is a video file as well so it's not compressible so even though we have compression enabled over there it's really not doing anything if we were to copy a bunch of big Word documents or big spreadsheets yeah gold it's worth noting that even though we got a great deal on these Drive shelves if all you need is a hundred terabytes of storage or less you don't need to do any of this stuff with a basic consumer motherboard and consumer chassis you ca
n do anywhere from four to eight hard drives pretty easily and spend a fraction of what we did here and just direct attach them all it'll work just as well with the free or inexpensive software we already mentioned and you don't have to deal with the noise of an enterprise-oriented solution that being said if you do want more capacity like this I'm one of those people getting one of these disc shelves you can probably find one on eBay if you're lucky for two three hundred bucks maybe even with t
he new modules you need an HBA which you can buy Chinese ones for a couple hundred bucks and then a cable that's basically it yeah and your drives and you're ready to hang out on our slash data hoarders yeah assuming that subreddit is still up these days you know what is up the Segway to our sponsor delete me your personal information is out on the web being passed around between companies like hors d'oeuvres at a fancy party and it's time to stop With a Little Help from delete me delete me will
help you find and remove hundreds of online profiles sharing your personal info so it can't be used by spammers scammers and even Slammers I don't know what those are but they sound bad in more extreme cases Bad actors having your data could lead to identity theft or fraud removing all of this information by yourself would be a huge task to take many hours but with delete me software and team of experts it can be done in minutes on average delete me finds and removes over 2 000 pieces of data f
or a customer in their first two years so if you want to get your personal information removed from search results on the web go to join deleteme.com Linus techtips and use code LTT for 20 off okay if you guys enjoyed this video I think the best one to go check out is setting up the mother Vault these are so quiet you can have that under your bed no
Comments