XR In The Age Of Vision Pro | SXSW 2024

hi everybody wow wow thank thank you thank you uh my name is Charlie thinkink I am the host of this week in XR I'm here with my co-host Ted schilowitz and our dear friend Joanna popper uh who has been a guest on the show uh over three times we have 190 episodes in the can we started talking to each other and recording it during the pandemic and uh here we are 190 episodes and 500,000 downloads later so thanks to everybody for that uh more people thank you more people have come up to me at South

by Southwest to talk about the podcast than they do about my books so I guess the books are kind of old computer books not in Vogue um well people seem to like what we're doing they like this organic kind of feel of what we do and what we try and cover what you write for fors and then you and me and roony who'll Maybe mysteriously show up in a moment or two um have to talk about yes Rony Rony abitz our other co-host uh will be out in a moment but this isn't this week in xar it's this week at Sou

th by Southwest at South by Southwest so um here let me I've got stuff prepaired uh Charlie Ted and Joanna take the stage Charlie is elegantly dressed Ted is wearing the same black T-shirt and jeans he's worn for 36 years Joanna is beautiful and comfortably stylish as a oh wait that part I wasn't supposed to read yeah so um let's let's talk about what we liked at South B yeah right so this is our News segment right this is the the part of the podcast that's the News segment so you and I have obv

iously and Joanna as well um are purveyors of these mediums right we enjoy them we live and breathe them and we've seen a couple of things we saw a lot of great things but a couple things that were real standouts right what was one of the standouts for you well we tend to lur about the XR experience area at the Fairmont so we don't see everything South by South West but it's impossible to drink the ocean as I say about South by Southwest you have fomo even if you're here correct especi especiall

y if you're here cuz it's just not possible to do everything but in the XR area the the problem with doing VR in a venue like this is throughput you only have 6 hours and you got to put through as many people as you can but you've only got room for one or two headsets the headsets need to be charged it's so you don't get to see very many things but there is an exception to that right which is fantastic and we're both big fans of it yes it's called the gold key and it's on the fifth floor at the

Fairmont and it is a room that is immersive it has three big screens uh but it is run uh locally by stable video and stable video is is they've got a series of pre- prompts qu of sort of uh Fractured Fairy Tales and then people add prompts to it and then you sit in the sort of Quasi theater around the screens and eventually you see how your contribution uh affected fundamentally the slideshow uh and I it that was something very few things make me go wow at this point after eight years of this bu

t that really have you seen it yet yeah they did a so the the the creators um Matthew needen Houser and Mark toos said they they had answer at Venice which is which to me is a what happens is you go into it it asks you a series of questions you type your answers and then in real time then then and then you go into the next stage where you're in a VR headset and in real time it creates a personalized VR experience for you and so that to me was a first glimpse of what the future of our entertainme

nt will look like where you have personalized entertainment that you help co-create that that the the um distance between creation and consumption continued to merge and so they had that at ADV Venice and then they also had at filmgate which takes place in Miami they had a piece which I think is a precursor to Golden Key where you became part of the um the part of the projection map and it sounds like that's what so go leave here and go over to the Fairmont everybody can see it at the same time

there are no throughput limitations and if we haven't given you enough Clues as to what we're talking about this is demonstrating some of the creative power of artificial intelligence so what Joanna and I saw with the toal mancer thing in six seven months watching the visual experience of how interesting and fascinating and kind of moment of crudeness and remarkable that was plug into 6 months later with this thing called The Golden Key and you will literally see like 25 years of progress in 6 m

onths it is remarkable terrifying and inspirational all at the same time we talk about AI on the podcast more than we talk about XR now so um roony our co-host actually says that AI is exactly what XR has been waiting for it feels like we need do we need to bring him out it feels like yeah but but right people are here to see Ron famous one that's is are there any other pieces you want to talk about that we uh well Eliza's piece Astra which is the followup to spheres is a must SE it's lovely lov

ely and Powerful it has great mix reality elements and that's really what what we're seeing now with the quest 3 although there's not a lot of content and for the Vision Pro which Quest 3 and Vision Pro will share a lot of mixed reality experiences and we just don't know what they're going to be like I did see something called impulse about ADHD in the EXP exor experience area um It's not finished yet although it's 30 minutes long but it was one of the best uses of mixed reality that I have seen

did you get to see chief that's actually on the Apple Vision I saw Chief yeah and that's a good segue cuz we're going to I think Charlie correct me if I'm wrong you've got the right we're here to talk about the Vision prois Pro that's what you guys came to talk about right so so let's bring out our co-host Rooney he's uh obviously everybody knows that he was the founder of magic leap and makeo Robotics uh he's an inventor he's a filmmaker musician and entrepreneur uh he wanted me to mention Sy

Sy B his new company um and so Ron out here let's give him a welcome come on Rony yeah your your chair's over there [Applause] [Laughter] sir is it safe could you really could you really drive a Tesla well could you drive a Tesla right now they got in trouble but they did drive it hell no so Anna Stern went skiing too all right there we go on the bunny slopes so let's before we um snark out here let's talk about what's great about the vision I'm GNA go get one to hold okay we have another got a

prop um I'll I'll go first um when you put it on for the first time it takes your first let's see how many people in the room have tried it yeah who's who's one one oh a lot of you all right so that's a pretty big number um it is the first moment is a big you know so uh but but um then it starts to to me kind of to be like an iPad and that part of it works really well yeah so so I actually have some notes scarily enough because I'm actually quite nervous about talking about this in public uh bec

ause I've been sort of working with this thing in various iterations for a little while and um it's now out in the wild you worked at Apple Ted's the biggest Fanboy I've ever met I am very much in apple culture and apple lifestyle like a lot of people are I I've started a number of companies and helped start a number of companies that were always connected to Apple in very robust ways lots of connective tissue so I want to start by telling you what I think is remarkable about this device in many

ways and great and I'm curious if you all agree with me as roony is there in in the process of using it uh as we go through this the first thing is that apple as a company this may be not a surprise to a lot of you is actually not a specialist in Innovation it's you'll be very hardpressed to find a product or category that Apple has started first what apple is excels at is refinement and polish they take things that are out in the wild that are kind of like we're going to argue about the refine

ment part yeah well that's what we're going to talk about kind of like not quite there yet and apple puts a huge amount of resources into trying to figure out how something should work and the great things about this device is that it is literally a Mac on your face meaning it brings all the familiarity if you're in a Mac ecosystem that makes you feel really really comfortable that you can fire it up and understand it right away and it brings all of that to your face the display Dynamics are fas

cinating and wonderful because it is a really smooth beautiful and the perception of high resolution is there later in the talk if we have time we'll talk we'll go a little Deep dive as to what's really going on with how Apple performs some really smart trickery around what you believe is resolution on your face that is different than the quest 3 and we can discuss that in detail but I don't want to spoil some of that well Joan has been using the Vision Pro and uh talking about it on social medi

a so I'm interested in your perspective remember this part of the show is good the good stuff you know I like to say positive things generally um so so there I would say there's five or six things that was really strongly impressed me when um in my in my time using this head said first I will say as an industry we're really really really excited this day has finally come um I think we've been you know many of us who've been deep in this industry which it looks like a lot of you here are have jus

t been you know we've been waiting and watching and waiting for for Apple to join to join the XR party right um and we've and you know we've known about all the Acquisitions that they've made and the aqua hires they've made and many of our friends who went over there we the joke is always we know what you did before they can't say what they're doing now but but we know the types of things they're working on so first I'll just say generally this was just an ex this is just an exciting time and we

're we're now five or six only five or six weeks in so it it really hasn't pretty fresh it's really really new so one you know in my experience the the mixed reality whether it's the perception or or uh you know as as Ted just mentioned putting it on was phenomenal it you know that the crispness the clarity the fact that you could see through it's not pixelated the colors the colors are bright you can you can see really well I mean r roone was able to walk all the way from here to here he missed

my hand but I think that was intentional so um that that part just blew me away and you know and I'm sure it probably did for many of you as well um second the ux was was you know it's one of the things that Apple does really well so the ux was relatively easy um maybe not completely intuitive but but pretty close to you know once you get the hang of it it's very easy and if you use a Mac it's very Mac like you get it yeah exactly um I will say the the the um what do you call the environments w

ere were phenomenal so I think that's been one of you know a dream of all many of us who worked in XR where you're able to you're able to go from complete um complete mixed reality where you see the world to go to to going into V sort of a virtual reality experience they're I'll calling it all spatial Computing but so you where you're in world you play doing email watching a video and you at all at the same time while you're surrounded by a beautiful Lake those are three great positive things so

I said positive so roony we're going to call on you now uh before we go to the other side and trust me we are going to go to the other side prepared uh tell us some positive things about what you like about this now remember Rooney is what we call the man in the arena meaning we're all talking about it Rooney has built and was the founder of one of the most meaningful mixed reality companies ever created and has gone through his own Journey so with that in mind let's let Rooney talk to you abou

t what's going on on his face positive things all good now roony nothing bad yet positive things okay um well I'm at middle of it right now I think it's happening somewhere but um John favro I think was executive producer of this thing called encounter dinosaurs which I'm running right now so that we'll be a large dinosaur I thought that was incredibly well done so good that was great so and and I know favro he's amazing and has been playing with augmented in virtual reality so I thought that pa

rt was just excellent um I'll probably leave it at that okay all right well then so uh I will say I want to add I I want to add another one or two things um one of the things that is exciting to me is that there are 34 million as of a year or two apple developers and you know our developer community in XR is much smaller than that and we have some amazing developers in our community but the Apple the Apple developer Community is just so large and with so much experience and that already with thi

s headset coming into the market developers from that Community have started who've never ever built before for XR have started to build for it and so you one of the apps Juno is being was built by somebody who would build mobile apps and you know all sorts of other apps and and it's already one of the higher rated or you know you from from the information we can get um it it it's one of the more used apps it's it's essentially a sort of YouTube esque uh video player since YouTube has didn't did

n't Jo uh join into the Apple the Apple Pro you have a couple of positives you want to bring into well I would just say that we've been talking about the putative Apple Vision Pro since we started our podcast four years ago so there's a lot of anticipation around this device I will give to Apple the Apple marketing machine in action is an incredible thing to behold uh and it's great for XR I love it when meta does it I love it when Apple does it let's evangelize this new medium because a lot of

people still don't understand what it is we think they do cuz we're you're here in this room you're a very rare person everybody else is kind of confused but less so now that the Apple marketing machine has had at it um the other thing is we were saying in 2020 I'll bet you the number one thing it has to be great at is Media consumption right cuz it like we said it's really a Mac environment and it's built by people that are not primarily thinking about gaming interactive they're thinking about

a media landscape so you know watching movies watching these sort of spatialized kind of videos being able to use the cameras on your iPhone 15 or the cameras in the device to capture spatial memories are pretty genius right I mean Apple has 5,000 patents sitting in that thing and they've been working on a version of it for a about 16 177 years so this is not like 5 minutes ago that they've been trying to think about well since the spatial memory thing isn't quite there yet maybe this is a good

moment to segue to the part of this show that we call dear Tim Tim now before we well let's bring her out there yeah dear Tim we'll do this we're gonna write a letter to Tim Cook together we're gonna go we're gonna go now this is what I warned some friends of mine this is a little bit dangerous for all of us so here we go now there's our drum roll now as we're doing our hoste change as we're doing our wardrobe change we have some confetti we also promised that we would ask the audience to take a

vote on if Rooney should continue to be wearing the Vision Pro and majority rules so if you think he should keep wearing it the whole time raise your hand and if you think you should take it off now raise your hand all right well r you know keep it on brother wait why do people want him to take it off they want to see Ron's face we just you don't want to see the red burn marks right now okay so uh see his googly eyes all right swords are out the swords are out so this is a little bit of our odd

like job interview for apple and we tried to figure out how would we get to Tim Cook in the most diplomatic way to communicate how much we love and admire what they have done with this device but that we also have a Litany of things that we think are very concerning and need to be altered repaired and advanced and maybe there's some people on the Apple team here because they come to South by so I have my little they find our resumes on LinkedIn Charlie I think they're jamming my system as speak

not and I'm going to spend like five minutes and then we're going to get into this debate okay so uh the first thing is I'm going to Ed this term that a lot of you I think will understand called balance does anybody understand the concept of the fact that if you build a tremendous amount of tech and Tech density on the front of a device and here we are in the year 2024 and this is not the first VR or XR device that's ever been built if you want something that's fairly heavy because what they're

doing with this Minority Report kind of Amazement takes a lot of tech and it's pretty dense so it's it's very weighty it's much more weighty than a quest 3 and other things it's all front-loaded if you look at what roony is doing it is not comfortable I can he can tell you and I've been wearing it for a long time it is not comfortable it bruised the bridge of my nose we did it the logical place to make it feel like it's floating is to take some of the weight and put it on the back of the device

and there's very smart engineering ways to do that in terms of like build a beautiful curved little soft battery that can go here when you need it here and it can be slid off or put in your pocket when you're sitting in your first class airplane seat and watching a get too hot what I would think the battery if you put the battery like there well yeah I mean these are all solvable problems but they need to be solved and they have been solved by many other VR devices or or or really if you are go

ing to do this right everybody knows this um it's super familiar to me because we did something like this so if you're going to move weight off the head I think yeah this is an important there's good bad ugly and inexcusable uh and I grew up loving Apple the one inexcusable here is if you're going to have the tether and you're going to have this anyway why did you leave the weight here right uh and it's weird cuz if you're going to copy that part do it properly so it's a little bit of a head scr

atcher to us literally right the this idea of an external battery pack in 2024 doesn't seem to make a lot of sense to us now if you're going to move the weight though it makes a lot of sense if if you're going to leave the then it doesn't make any sense yeah so that's the big one right another Computing weird thing about it and I I will say I mean I I I worked at HP so I also worked at a company that built headsets and we we launched you the HP Reverb G2 and and many other head and the omnis and

and many many other W winning headsets um and so I I do think it's you know it is it is interesting and surprising choice to have to have a pack um that said for Mo most of the people that I've spoken with because there because and you know in in the quest three the quest ecosystem about 70% of the content is games and here about 70% of the content so far again you know six weeks in is is apps or um well just apps and then another like 10 10 to 20% is is entertainment so you're not the way that

people are currently using it they're not moving around as much and so they're you know they're often using it for email or or work or productivity and so they're so it doesn't interfere as much as some other head you know as it would with other headsets not moving your jacket so they can see so so people haven't said ruining the plan yeah so so people haven't from the users I've talked to they haven't been as concerned about the pack as I think people like sort of Insider baseball is that's ve

ry smart how much of that is Brand Power though I mean we have to give Apple I think a lot of what Apple does isand it's an amazing brand with huge brand power that if you love it it doesn't almost matter what community a Hu which which I think is the you and those of us who've worked at other companies building headsets like the or or created companies that that's one of the reasons of excitement for them entering the ecosystem because because it will bring so many more people to experience spa

tial Computing for the first time yes so there so there's two more pieces of this puzzle that we are kind of these are our dear Tim moments the other one is this so there's a lot of smarts in that but there's a lot of not so smart and the not so smart is every time I would put somebody in this to show them because of my work I would say okay now please understand and listen to me very clearly and watch my lips when you hold it when I give you the device when you put it on your face and take it o

ff do not hold it by this part only hold it by the metal part because it's $44,000 and it will drop and break if you hold it by this part now great idea the magnet but it needs to be a much stronger magnet if you're going to do that right uh the light seal works well but it's a problem do you guys agree oh yeah I mean and then the last piece and then we can go into the open debate is what I would love to know from Apple is how much weight and Dynamics go into the kind of groovy cool sort of fron

t display that gives you kind of the creepy eyes and that's fine and it was an interesting design choice but at the end of the day if it added weight and cost to this which I I suspect it did I don't get it I just feel like I want the thing to be as light and balanced as possible and the gimmick tree doesn't buy me anything right now so that's that's my laundry list it's not very long but it's significant for a first gen device and I think all those things need to be addressed and in a very posi

tive way so hopefully Tim and other Engineers like you have to imagine somewhat an apple somewhere in the in the sort of Universe of everybody reporting up to Tim would raise their hand and go yeah but you know if you put some weight on the back it won't feel as heavy on the front how do they miss that I don't I just don't understand I think they probably thought oh well you know the accessory ecos there's two straps aren't there there is there okay so you're right that's a good there's two stra

ps that it comes with there's a second strap it has a top strap it doesn't really work it tries to lift some of the weight off so maybe it was an afterthought that someone was like you know we better ship with a secondary strap cuz I mean this strap is cool the design work is great on it it's very comfortable when you tile it down and if you're sitting back like this it's fine but if you wear it like this for any length of time it's not if if it's a productivity device having to tilt your head b

ack is OB is optimal you're postulating well I just think that I think one of the that that making things for your face is hard because all our faces are different and so it all it has and so if you ask you know 10 different people there'll be a whole bunch of different answers on whether or not something is actually comfortable on their face and so you I'm you know Apple I'm sure did a ton of user test ask the audience right A lot of people have worn it like I don't feel like I'm saying anythin

g that's so hyper radical that this is the first time you've heard it or felt it yourself so if you feel like it could be better balanced and less perception of heaviness and weight on the front of your face I guess just show by a show of hands so you see not their phone was it comfortable are comfortable for there are people for whom they feel that it's very comfortable for their on for them so Rooney we haven't even not very many we haven't even gotten you going some people yeah Roy go roony c

ome on man let it rip well the the reason I wanted to wear it um first of all I want to compliment before I go into the rip part like anyone who works in this kind of engineering this is really hard stuff uh it's it's I would say that engineering this is actually harder than working on something like space cuz we already went to the moon we already did all those things no one's ever fully cracked this problem uh the number of sub components and subsystems are actually and what we did at Magic le

ap and I'm sure what's happening in apple meta it's like equal to the number of subsystems in a Saturn 5 it's like that insane and the number of simultaneous problems you have to solve are worse than electric car problems it's just like one of the hardest problems and it's why some of the biggest companies in the world have spent somewhere between 4 to 80 billion each to date and we probably spend north of 100 billion each so the you know one of the criticisms is if you're going to spend that mu

ch like as a startup raising a few billion is like the outer edge of what you can hope for and when you compete with people spending 40 60 80 billion you do expect more than this so I think I think you look the order of magnitude to spend to date and the fact that we're not in 2012 but it's 2024 I think that especially if you've been in the trenches I think there's a higher expectation so one of the things that I wanted to wear on stage and talk about was I feel incredibly disconnected from all

of you right now like I don't actually feel like I'm here I could be back home sitting on my couch because the the video that I'm seeing of everyone is worse than sometimes being at home on Zoom because it's much more blurry it's much more grainy it feels like I'm looking at the world through gauze and it feels like I'm not even here it's like there's there's a Strang I feel like I'm a I could be Tes in in Mexico or Paris right now so that part of like disconnecting me from the reality is like I

think breaking one of the tenants of augmented reality I think it's got some really good things as a virtual reality device but I think the interesting thing was I wanted to understand like what is the best in the world video pass through AR feel like today versus true see the world direct if you actually see the world directly through AR and there's a few companies that have done that including what we did at Magic leap there's a magic in actually seeing the world through your own eyes with Cl

arity and that magic is completely not present when you're looking at the world through gauze and this is like much much better than like what meta has done with the early black and white stuff so this is like the best after tens of billions have been spent so that part I feel like you have to you've got to go back to this I mean this has got to be the endgame you know where we could see the world directly this is the foreign Factor this is the weight and we've got to get on that path and I hope

Tim and Mark and others who were in The Game including the company I found i f they all like realize we've got to go to this North Star this this is like going off the road into the wrong direction even though it has some amazing components this is never going to be all day every day for the world it just won't so and cuz your dog in the hunt was all about pass through mixed reality that was the device you designed it started see through see- through see-through mixed reality like not it's not

video cameras that you see the world it's your eyes that you see the world and that was in a very important belief structure that you have and still hold to like look at the Gap this is about 110° diagonal field of view we built a pass through with 30X less dollars that got to 70° there's a gap of like 30° and you're telling me with another $40 billion you can't solve that like I just don't buy that so I feel like like hopefully this is a push because we all want this to happen I think we're on

stage because we're super enthusi I wanted the field to happen because I want to be able to make amazing I'm sure most of you you don't want to make the film projector you want to make amazing new things like yesterday we talked about story worlds I want to be able to make Amazing Story worlds on incredible devices that look like this um we want a billion people using something right so that's your dear Tim moment you want to get that information to Tim and Mark and and everyone in this space li

ke we want this to be amazing but pleas get back on the North Star SAR also and Sundar I mean they had their chance I think they're coming back with Samsung well it's comment on them it's clearly a developer version yeah there's no content for it if you buy it you're supposed to make the content right and it's been marketed as something for consumers and there are lots of people who spend $5,000 on Apple stuff reportedly 200,000 people yeah so and and apple is touting that as a huge success and

claims that they can't make them fast enough well I think it fits in this really in hybrid right we all know that it's a developer product based on its price tag and it's deliverable we also know that there is this fandom around Apple that is not price sensitive and many people that can afford it that don't use it for any professional version but want to have the coolest hottest toy around like we often joked in our little worlds that there's a mapping that if you want a high-end Tesla not a thr

ee but a highend one you've bought one of these it's a direct onetoone correlation TR you bought if you bought a cyber truck you bought four of these things yes so I guess that begs the question if this is the developer version what is the iPhone version that we all should actually buy it feels like the price point be interesting Joan and Rooney should be sub $2,000 cuz that's kind of if you didn't get it on a contract which your iPhone actually cost you if you got the pro version so it's a coup

le of grand um it feels like over time because apple is in the refinement business right they used to kind of keep it all secret until it was super refined this is 80% refined the last 20% is a couple years away we see the nextg sort of version of this I think I I don't know Ted I mean if their direction is this we're not a couple years away I mean the question is what's going on behind the scenes and is this just something to get out there I I do have to say something cuz like I don't know how

long we're into this like we're we're halfway six2 I played football in high school um this thing is on my neck like it is heavy um and I just don't I'm a biomatic engineer I'm going to rail on this for like two minutes we obsessed about that uh when we were building it we were like so worried about biomechanics and anyone that has was been in the field for the last decade uh and the virtual reality has been around for like decades like there's so many devices you could wear and understand what

it means not to the neck of a 62 big guy but a 5'3 you know 5% female you know you're 120 lb you're 53 this is not something you should wear in your neck and if it's going to be like I'm going to be productive working all day what what are you thinking what about the other 50% like the whole human factor side it's like everything I grew up loving about them is completely missing in this design decision I just don't understand no we completely agree and this is kind of why we did this we knew thi

s was a risk we knew it was a little goofy I'm getting wait I'm getting a message you were about to be cut off from the Apple but we decided that the best way to at least get the word out and maybe this can get through our friends at South by or through some of you to somebody at Apple is like maybe you should watch that thing on stage Tim and listen a little bit in a very positive way you know we're not trying to bash this thing to death we think it's great in many many ways by way T I think I

think being able to say this with with credibil like having built stuff like and knowing how hard it is and knowing how many difficult design choices this wasn't a hard one there's so many difficult design choices but understanding biomechanics is one of the first things you do so that part is just incredibly confusing they'll never answer us on that one but hopefully someone hears it and the next gen will not have will not have it yeah or do with who it was tested on and what their feedback was

right like what all of the user testing that they did 101 human fact we we hired a woman she was amazing who did human factors for wearable devices like the best in the country and the first thing we did was a study the 5% female to the 95% male the biomechanics of the cervical spine and we thought oh of course apple and others who we thought we were competing against they probably were going to hire all of those people so we better like over rotate on that this was like did somebody forget to

do that in the requirements document like this for a geeky engineer it's one of the first things you would do in designing an XR system map out the weight allowance on the head of the 5% female and you can't be worse I do know that there is already kind of an underground ecosystem that is working on a modification there's a lot of developers independently building it's a whole opportunity for per so it's coming now will it be as refined as an apple first py thing product first party thing no way

but at least it'll attempt to start to solve the problem and maybe Apple will get the message and go you know what maybe we should do that I say one good thing though because I don't want to just think I'm I'm railing um I've got a giant screen open right now yeah and if we didn't have this like Gauzy film of looking at all of you where you're all like Blurry and mushed the giant screen is actually super high res so the non augmented reality the virtual reality part is bright and amazing so it'

s a really good but very heavy virtual reality device and they don't want to call it that but that part's awesome like I'm to the Moon in a second I'm just going to be on the moon with a huge screen I'm going to watch like super bad as this is going on and you guys will all just fade into the moon but that part's cool the Virtual Reality Part I found to be really awesome except for the fact that it's hurting my neck the rest of it as a VR device is great so it seems like whenever uh the qu uh Th

e Vision Pro 2 comes out that we will be in an ecosystem where we have um a a duopoly between meta and Apple so one question is is there room for a third is xreal even going to be able to make a sound right xre is I don't know $150 million $200 million company going up against apple and meta right now so what he's talking about and Charlie refers to this as what they call screen reflector devices or screen extension devices so you connect your phone today with the cable very soon because of our

friends at Qualcomm wirelessly to a device that is lightweight it's typically using 1080p what they call bird bath Optics so you get kind of what roone is looking at with a a um an optical P like you know just like looks like like 350s and it works great now it is nowhere near the minority minority report experience of this this is Magic this is full on Magic that's for uh media consumption right but it works great and it looks highres and it's great and it's very inexpensive right the thing you

and I were talking about at the bar last night which is an interesting piece to kind of bring into the puzzle is do you believe and I think roony and I might differ on this a little bit and the audience would be interested in Q&A part do you believe there is one Dev one device to master them all or do you believe that the ecosystem that we already have with computers you know we all have mobile device form factor iPad is device form factor laptop form factor and very often we use if especially

if we work in some industry that requires compute we use Advanced desktop Computing right and we have kind of four or five devices every one of us and we use them almost on a daily basis so the question had enough roony finally yeah they wore them down the question for for Rooney Joanna and Charlie and eventually for you guys too is do you think one size fits all or and I saw you know a lot of you shaking your head already or do you think there'll be like a bunch of different ones that you'll ha

ve for different purposes just kind of like our compute ecosystem today that's the interesting question well as there is with phones right you can buy one of these bad boys for $99 and get the internet and a lot most of the benefits so that's one end of the spectrum or you could buy the 15 Pro all a cart or off of a plan, 1500 bucks something like that yeah so it's a big Spectrum yeah so don't we think it's going to rep it's just about like what kind of screen do you want and what's the proper e

nvironment for the screen well and it's the use case right or the jobs to do as they say so I I believe that we'll continue to have a variety of different different environments that or or different different Hardware that we use for different moments different moments at different times I'm going to argue with you guys though yeah go you you have the counter so I I think something like this that could just sit in your pocket that will have powerful compute and we know there's going to be Wirele

ss connectivity we were testing it companies that could spend 40 billion more for sure can get there and what I have on right now the wraparound is better field of view by far than what I was wearing um and can you get immersive at retina resolution or better in something this thin or small absolutely um I know we can and I know we were doing it with less money well you've seen companies with t dollars can totally get there so the question is like are they putting things out there in the middle

period why I call the Gemini period before we get to Apollo but Apollo cannot look like that right like late this decade into the 2030s I think if you got something like this and something like rayb bands um and you're going to get re a resolution wirelessly talking to a small thing in your pocket and your watch there's no reason to have any other devices you'll be able to do everything on it and I think that's the beginning of what you see on the Vision Pro The Vision Pro from a software side i

s showing if you get to this form factor it it's your phone it's your tablet it's your computer it's your everything so so let's let's shift the conversation a little bit to Ai and you're talking about the 2030s and so here's a an alternative version of a wearable immersive helpful co-pilot or LIF logging device if you will and it is sensors right whether they're in your glasses maybe you don't wear glasses and you have it on your lapel or and so it knows where you are it knows what you're doing

it knows what time of day it is it knows your habits and there's an audio connection but there's no display what why do you need the display at that point yeah and that's already like today proving that out right that the audio system by the way the audio on this is remarkable it's clear and beautiful and I think Steve Jobs would be proud I think Steve Jobs would not be proud of this in my opinion I think he would be telling people keep on it keep working on it keep iterating on it don't releas

e it like that I think what an interesting interesting thing to think about at that point think about the ecosystem so qu the Oculus Facebook meta have now have uh you know all the same company but under you know different names and different points have released 10 headsets already in the market and had about 85% of of the market and you know if you think about why they went in so early you they they I think their thought was that there was an ecosystem of IOS and Android and that ecosystem dis

dis intermediated them from being direct to Consumer and wanted to make sure to have one of those two poll positions in this next wave whether whether it's the only device that we that we use or whether there's multiple devices they want it to have that direct ecosystem which is why they put in such heavy investment so early and have already been out into the market now with 10 devices and so Apple came in now you know they could have come in earlier they could have waited to the I mean they're

they're they're coming in from a winning position where you know like they they they have a a position to potentially lose right they're not they're not trying to get at a position whereas um whereas the Facebook or meta ecosystem wanted to make sure that they were there and they were ready to to to sort of to potentially take one of those first those two positions assuming will have well that that it looks like an a Mac and PC or looks like an Android and and um so R is the guy sorry uh Ron is

the guy who said AI is what XR has been waiting for so now you're talking about 2030 we're talking about AI you're talking about all day every day wraparound glasses give us a picture of what a day looks like with such a device yeah it's a good question and I have a bias Scaris I'm working on AI right now in synp but um one of the things that I totally see happening you could prototype it now but I could see this mid to late you know late 20s early to mid-30s first of all a device like this eve

n though this can do it you're not going to use it all day will sense everything and I could see everyone clearly and you imagine go from Full transparency to full opacity and everything in between so imagine we just have that so how does AI play a role now imagine an AI that like really understands me and my privacy and my data is assured it's not streaming to a big company it's like yours so now it knows you and knows your likes and knows everything around you understands the people around you

but you've got this dat of privacy when you have this on it's like your IQ and your awareness of the world and your poers are going to be 10x 100x everything you do in your profession is going to be hyper Amplified when you take take it off it'll be like someone who takes off their glasses and has really bad vision and everything's blurry and you can't see you put it back on and it's like you know everything people ask you things you'll get answers in real time your awareness now think about ho

w good AI is right now in 24 which is astounding it's going to be completely insane by the end of the decade and into the 2030s we be unrecognizable to the kids who are walking around wearing things like this whose IQs are orders of magnitud more than we are to we'll feel like Neanderthal even worse like the separation between the AI Amplified people and the people who are just normal biologic whatw will be completely different I think the best way to consume it will be wearing this because it s

ees what I see it senses where I am it knows what's happening I don't need to wear a pin it I it'll have the same sensing but better because I'll know exactly where my eyes are where my attention goes the key thing is who controls that AI is it decentralized is it centralized is there privacy and control are you submitting everything about your life to a major power or do you control it I think that's going to be the big decision not if it's going to get there I it's 100% going to get so roony w

hat yes you should clap for that W so roony what you're referring to and I think whether wittingly or unwittingly everybody in the audience knows this already is that we have this little pocket superpower thing that there's one you know on Charlie's lap there and we all know innately that it's power isn't in the device its power is in the giant computer in the sky right or the many of them that what we used to call the AR Cloud when we say giant computer in the sky what are we talking about here

well we're talking about today's version pre AI moving into AI which are these servers that live all over the world that deliver all that it was a Norman Green bomb green bomb song that needs to be redone the giant computer in the sky computer in the sky going up to anyway so we know this innately we do this every second of every day almost is we ask our little pocket computer to call up to the mother ship and tell us what's going on and what you're saying and I think where it goes is if you ca

n dewe the entire system by just allowing it to be the bare components that need to drive the visual and audio and everything else is the feed for similar to this you can get to that device compute at the edge yeah I mean everything moves and you know what until that point if you're going to t something with a little cable put all the weight down here you can have this in between we 100% agree so uh as most of you know at this point in the South by Southwest there is a slido feature to the South

by Southwest app um I wasn't going to do a lot of questions we have 15 minutes left but Hunter Lane gave us such a good question and it's so appropo to where we are in our talk I thought I would just read it was this good did you enjoy this nonsense is it a good idea to there you go take the question though yeah how do you from from how do you expect XR platform content strategy to shift in the age of AVP and spatial Computing and this is a great question because right now there really is no co

ntent so where are we going with that and is it as much of a shift or do we already understand an ecosystem in IOS and Android that people all around the world build all kinds of apps very few of them actually reach what we call escape velocity right and actually become marketable and profitable anybody cares about but there are hundreds of millions of apps when an iPhone first launched faite what was your favorite my favorite app of all time yeah you know the one I'm talking about oh yes sorry

we do have a little okay so this is another dear Tim moment um roony and his team at weda and there's our friend Richard who helped create it his team at weda Workshop uh for the small amount of people that saw it there was a mixed reality entertainment gaming piece called Dr G got Dr gbots did anybody get a chance to play it just a few of you right at leap to Dr gords BBS Dr G I'll call it for me particularly someone that does this a lot it was one of the most important and still today one of t

he best created and most important mixed reality interactive pieces ever made it took full advantage of a spatial universe and you lived inside a video game Ernest Klein would be proud to see it and we would love to see it on the Vision Pro yes we would I'm making the open call since most people didn't do the experience just uh as an aside it it used the room tracking uh in the headset to sort of create portals on your walls and on the ceiling so the game really was all around you and coming out

of the floor and the wall and you could walk up to the wall and look into the portal and so it was a great use of space which we don't really see in any other medium I never know where come toal Computing before but it was it was definitely a spatial competing experience so another question question we got another question uh is there something specific where where did that go is there something specific to AR and VR that is so hard to develop for that one has to release a prototype as Apple di

d what are those unique challenges that is also I can't see whose question that was it's a good question let's do it yeah so it's a great question we have a lot of time I'll jump into it really fast one of the issues is that you have the human body so I'm a biomic engineer and you need to interface with the the the brain our our neurologic system or opthalmologic system the weight the spine all of that you're putting something on the head and it is a brain computer interface it's not a normal co

mputer your mobile phone is a computer a tablet is a computer this is a brain computer interface ignoring that connection between Computing and biology is what we're actually talking about here a lot of companies ignored it it's very hard to ignore in fact you cannot ignore it and be successful in the end that's one of the most difficult things the other piece it's like a satellite because you have so little weight that your body wants up here and because your eyes have evolved into working into

what what what's outside of you it's called a light field lightfield physics is completely non-trivial your eyes evolved into lightfield physics that is not what's happening this heads set generating a True Light field that your body can respond to appropriately is insanely difficult it is one of the hardest most difficult problems in Computing it's just a non-trivial problem and that's just like one of about 290 things you have to get right at the same time to get to what we're calling Apollo

so Jeremy is asking about killer apps I've been saying for I don't know years that what XR needs is our Game of Thrones moment or our Soprano's moment where people are like I have to do this no matter what the cost is and we don't have that yet and I don't think e as great as media consumption is on the Vision Pro I don't think they have that either I would say that that's that's potentially why it was was released at this point you know there to cultivate developers Cate yeah to and you know it

's an interesting choice I think you know you could could have been done like publicly or could have been done in develop so maybe one of those developers has new Instagram thousand apps on on you know in in the first six weeks and so we're still early but you know 200,000 people have purchased it some percentage of you know decent percentage of those developers will will'll be every day we'll be seeing new things come out and hopefully get to that that uh I I think killer app I think potentiall

y the killer app even though you may argue it's not an app is what will evolve with spatial photography and spatial video memories with the device right your baby's first steps a very important speech an important world event that is captured in a spatial environment not just 3D but actually a spatial environment because with the spatial capabilities of a device like this and other things from our friends at meta and others there's a whole new realm of what photography videography and cinematogr

aphy will be that I think will be very important if if you haven't played with photo grammar the new photo grammar Tre apps and I'm going to call out polyam because I use it all the time all of a sudden we are able to capture not a flattened reproduction of what was seen through the lens but we can now capture the space around natural geometry exactly so I think that that'll be extraordinary and what this does also and you know again somebody who worked at a Computing company is that it drives p

eople back into the Apple ecosystem so the number of developers I've talked to you know traditionally uh VR developers were worked on PC the number of developers who were saying oh now I'm buying a backbook now I have to bu the new Apple 15 because I want to be able to do the spatial capture so you know or now I'm now I'm moving from Google photos over to Apple photos because I want to have my my photos there at my fingertips so it's move it's also continuing to reinforce their ecosystem which i

s you know the point of the point of having these companies with so many products for for the company can I one quick thing to the I I think a really interesting killer apps going to be um watching a movie on it if you remove the weight part is kind of amazing um and I think if you take the next step which is like sitting on the moon which is full immersive mode is amazing and you go what happens Beyond movies we have a talk on that yesterday I think story worlds uh which is like a parallel thin

g to games uh it's not immersive gaming is going to be incredible like there's going to be directors and and favro started to get at it like he gave me three minutes of a rectangle of like a story that felt like the beginning of something really cool and then it ends like I want to see that whole thing emerge the sort of cinematic reality spatial movies um and I the dinosaurs are in your environment well they're in your environment but also you can interact with them it's not it's like how do yo

u tell a story that's not just that 2D world and it's not just the Avatar 3D movie in the IMAX theater but it is potentially all around you spatial cinema this device could open that door and you got to give like the greatest directors big budgets to really understand what could happen yeah and there's a few nuggets in the device now that you can see where that's going there's this woman that does this Highwire walking stuff and everybody I put in it is like whoa like this is crazy because the i

mage is so good it's very beautiful and cinematic and resolution works at that level um the encounter dinosaurs thing is good and then there's actually a longer form thing in the Apple TV platform that actually is like a series of that um and the Alicia Keys music video thing is also really good because you feel like you're at the recording stage that's 3 really interesting or or the sporting events yeah it's just it's really interesting but again here we are talking about like sort of tradition

alist media kind of taking a little bit of a step forward right but this is how technology succeeds because it takes something we're familiar with something we're already doing like shopping or sending letters and it makes it way easier and remember when you're in the device what's kind of great about it if you're a Mac ridiculous Mac guy like me is everything that you use in your Mac is still happening on your face so when your texts come in there they are like you don't have to do some weird t

hird party craziness and try and link your computer and your phone and stuff it's just all automatic when you go into any environment that you've ever been on in Wi-Fi on your phone it automatically knows because it knows you it automatically links to that Wi-Fi all these little apple e things that apple is just better at the refinement level is great when you set it into this mode where you want to mirror your computer into the device it's completely seamless it's perfect yeah by the way is a t

ough part for meta right really tough because the the the Android ecosystem could in theory more fragmented bring you like that no but but the you know if you have an Android phone you can go there but like apple is going to bring you phone tablet Mac there that's that hard lock of the ecosystem C they spend a lot of time on it and it works great meta I mean meta does capture a decent amount of our time between Instagram and WhatsApp and Facebook and I think those are their M they they did workp

lace but they they haven't integrated those the as seamlessly into the quest experience that's what theyve done a much more clunky way but they do have an opportunity there but they have yet to do it yeah they have they're not the automatic windows iOS get we're going to do speed questions speed questions well somebody's asking about Enterprise XR and uh whether the Vision Pro will have an impact on that and the answer is it already has yeah so Medical Healthcare they're starting I'm really hopi

ng that this is going to read people were doing with Hollow lens in the past we've already seen articles about surgeries and and things like that being used for with Apple Vision Pro pull your neck down the whole time it's going to be problematic the industry has been trying to scale has been trying to scale Enterprise for the eight years I've been covering it and it is happening maddeningly slowly even though the benefits of it are really clear so it shows how certain industries are G to evolve

generationally you're not going to convince them to change right now because they're too busy doing other things and despite the best efforts of Accenture and ernston young and other companies that have really leaned into XR their clients aren't doing it and it's actually killing the XR segment uh the Enterprise segment of our business which for a long time was the healthiest part yeah we're waiting for that moment right and maybe this is part of that moment this an interesting question about j

ournalism go ahead read it how it's from Anonymous thank you Anonymous thank you Anonymous how can journalism evolve in this new landscape what kind of formats can you imagine that's that's your world Jo you should answer that I think it's a it's a super interesting question I IM you know we've we've moved a bit for better for worse often for worse but to Citizens some level of Citizen journalism where people are getting their their news from people posting things in social media and and I think

I can see an interesting an interesting use case where people capture in spatial you know people with their apple 15s and 16 17s 18s capture capture 3D moments and then you're able to go in and see them um and in a world where things like deep fakes and synthetic media can have a big impact on our understanding of of the world and politics having something that's spatially captured you know maybe that that you we'll see what kind of AI can be can be um layered over it but po potentially could h

ave could bring you into moments and and into Spa to spaces and and let you know that they're real would be a sort of my optimistic point of I like this one this may be our our last question cuz they're going to kick us off at some point uh seems like meta is going towards glasses forign factor with AI powered Rayban do you see it converging with Quest that's a good lead into we've talked a lot about Apple today we've dissed on them hopefully in the most positive diplomatic way we attempted to d

o and we still left our resumes are on LinkedIn we still left a few things off on the list that we didn't even talk about they still need to fix talk about that after um but I want to call out our friends at meta in the VR World Charlie and I often talk that without meta and that huge amount of capital they could putting into that long game we would not be as far along in this equation the quest 3 is an exceptionally good device especially when you realize at retail it's only 600 and something b

ucks and it has mixed reality capabilities and it's fairly lightweight and it is and you can buy an accessory or it for today that's a first party accessory to balance it it's 500 yeah let's call it call it 600 bucks um you know but it's definitely a lot less than this right by order of magnitude 7x is the the video that Zuck put out and it is really really good at a lot of things if you're willing to kind of live and we just say it's a little more window zy you know it's not quite as refined bu

t it's an exceptionally good device now the thing I will mention to answer that question and then I'll let everybody else respond to it too is meta has teased now some of their employees that report closely to Zuck about this magical new device that they're working on that is the end all be all and they literally talk about it like the Holy Grail like they're working on this thing that is so much better than a quest and so much better than a Vision Pro and so much better that it will cure all th

e ills now will we see it do you think that they're really on that track maybe this is something for roony um I think partially they're working really and they spent a lot of money to figure out how to get something like kind of like your visor experience but will they get that right in the first iteration I I think with the amount of money they've spent you could get there with about a fifth or sixth of the money so I would hope so right I mean like uh I think Matthew ball estimated North of 60

billion some time ago so they're probably at 80 billion right now at 80 billion yes if they don't uh they a whole bunch of people there need to be replaced with others who can get it right because I know Mark will keep fing until he gets there right so I'd like to thank South by Southwest I'd like to thank Blake caminer who is the head of the XR uh exhibit and uh talks uh I'd like to thank our listeners uh and especially our sponsors zapper uh who makes it possible and uh I hope you guys enjoy

the rest of South by Southwest so if you haven't been on this week in XR every week we do this this will be this week's show and every week we get on Rooney Charlie and I and we dish it up a little bit every week so please listen shirts for the audience oh yes we have giveaways swag we're going to throw shirts we have swag you've got seconds to want one of these collector item shirts somebody over there and you yourself want to ask Tim oh too far sorry all right I know you I'm get I'm trying to

get it to you there you go we go yes dear The Rock and thank you guys the rock thank you guys thank [Music] you [Music]