Discover Stanford for You: The Intersection of Artificial Intelligence and Education

and good afternoon once again uh I'm Nate Boswell I'm assistant Dean for continuing studies thank you for joining us today for discover Stanford for you we're excited to bring you this event which is being co-sponsored by Stanford continuing studies and by Stanford's office of external relations and features two renowned Stanford researchers Victor Lee and Denise Pope after opening remarks from Dan Schwarz the dean and faculty director of the Stanford accelerator for learning at the Stanford Gra

duate School of Education Dr Lee and Dr Pope will discuss the dynamic Rel relationship between Ai and education with about 15 to 20 minutes left in the hour we'll feel questions from the audience and we'll prompt you to submit questions before via the uh Q&A zoom function you can do that along during at any time during the program uh the program will close with comments from Megan Sweezy Fogerty senior associate vice presidents for Community engagement uh and to kick us off I'm now going to turn

it to Dan Schwarz who as I said is the dean and faculty director of the Stanford accelerator for learning at the Stanford Graduate School of Education welcome Dan and now we'll turn it over to you all right thank you Nate and thank you everyone for coming um we we've known AI was eventually going to make it to education uh but nobody predicted how fast it would show up once it once it broke through uh this is very exciting from the perspective of The Graduate School of Education uh this is kind

of a great great thing because it's forcing new questions people always have lots of assumptions about what education should be and is and suddenly this showed up and all new questions and so the there's getting a lot of attention uh to education which is good and what one example of the energy around this is uh we the accelerator put out a seed Grant which means uh faculty staff students can compete for money to study uh develop Ai and education and this Grant had more applicants of any other

seed grant that that's happened at Stanford as far as I know we got uh several proposals multiple proposals from every every one of the schools except for the law school uh which surprised me because these new tools are going to have a big impact so this is very exciting it it raises fundamental questions about what students need to learn and how do we go about teaching them uh at the same time there there is a concern on my part at least which is uh AI could make us very efficient at what we've

we're currently doing and some of which isn't necessarily great right so AI is based on the past to some extent and so it could get good at automating the past and we may not want to just replicate so my My Hope right is that uh the re presence of AI will lead to new Innovations to better ways of teaching uh to providing feedback to helping people discover and achieve their dreams and uh I think I think this is a place where a university can do special special work because we have the space to

really think about things and try out new things without tremendous Market forces pushing us so today you get to hear from two of my favorite colleagues uh Denise po and Victor Lee and they'll tell you what's going on out there so Denise is a senior lecture at the Stanford Graduate School of Education and the co-founder of challenge success which is a Stanford Affiliated nonprofit that works with schools and parents to improve the students well-being their engagement and their sense of belonging

importantly maybe most important to me is she is my co-host she's actually the star but I call her my co-host on the podcast schools in which disc discusses the research behind current issues in education the other speaker is Victor Lee he's a professor at The Graduate School of Education and his scholarship focuses on uh studying and designing learning experiences and resources for data literacy uh for K12 data Science Education and uh for artificial intelligence literacy and importantly he le

ads our initiative in generative AI at the Stanford accelerator for learning so uh Victor and Denise recently collaborated on Research that was exploring what students and teachers are doing with tools like chat GPT and now you're going to get a chance to learn more about that work today so uh Victor and Denise take it away thanks Dan appreciate that intro Victor is sharing his slides I know some people are still coming in so welcome to those of you who are still coming in it's a big group so it

takes a while for folks to to get here but I will turn it over to Victor to start us off thanks and Denise uh do me a favor as you always do and everything's looking okay with zoom we always like to check everything is looking good excellent well that's already a flavor for how Denise and I interact we're really excited to have you join us and while we talk about the intersection of AI and education and in preparing for this um Denise and I had some conversations we wanted this to feel Lively w

e wanted it to be informative but Lively um first we talked about oh maybe we could do this like a podcast but it turns out that Dan and Denise already have a podcast so we couldn't do that um I suggested that we do something like hot ones that YouTube um show where you take turns eating really spicy chicken wings and answer like more and more revealing questions but ever since the one chip challenge um I think the university put the kabats on that so we are not going to be enjoying chicken wing

s but we do want to keep this really conversational we wanted to to um really speak to a lot of things that we hear repeatedly um in hallway conversations or at talks or presentations and so what we're going to be doing is sharing things that we hear and what we here at Stanford have um to say on that particular matter so to illustrate um Denise if you don't mind yes so Victor you know we hear a lot of different ways of referring to AI some people just say AI some people say chat GPT some people

say llm which stands for large language model um and I know those are not all the same right so walk us through I think this is going to be a good lesson for folks out there who don't know the differences sure and I can completely understand you've been hearing a lot of letters like GPT AI llm um and it's getting really confusing as to what all those terms refer to so I'm going to just share very quickly a a short primer um that I've uh that people told me have been really helpful for them um s

o first off GPT GPT is everywhere um you know it's it's uh three letters that we didn't really think would be put together and not everybody knows what they stand for um but it stands for generative pre-trained Transformer now nobody's going to be quizzed on that or need to remember that it's completely fine to just say the letters GPT um but what is that exactly and how does that relate to all this Ai and chat GPT and llms and such well we know the internet is um full of text you know there's W

ikipedia and there's blog posts and news articles and and many things that are on there huge huge amounts of it and so um what we've been able to do over recent years has been to use large amounts of that text as training so as input into a mathematical system that um we would call a neural network um and that basically does mathematical calculations that helps to determine what words would tend to occur a lot with other words and um that we would then refer to that sort of uh Network that has b

een trained and knows what are the probabilities for those as a large language model so an llm and the llm that has gotten a lot of attention and if any of you have tried chat GPT the free version will use what's called G PT 3.5 and so that's basically the fancy name for the very specific um large language model that chat GPT uses now if you pay to use chat GPT then you may have access to GPT 4.0 so that's just like hearing oh I use OS 10. uh 6 but now it's going up to OS 10.7 um so it's the sam

e sort of a thing where it's the underlying guts of it but to make the distinction here um chat GPT is effectively what we'd call an AI chatbot um so think about this like a car um the large language model in this case GPT 3.5 is the engine but the car itself is the chatbot so chat GPT is a very specific model of car and it runs on and is powered by this GPT 3.5 engine but you know has all these different features on it like oh it's going to appear through a browser window and if you ask it if i

t's human um it's going to tell you it's not human um if you ask it things that are uh from recent history you'll say sorry I only was trained on stuff prior to the year 2021 so it has a bunch of different rules that a program would be um expected and and uh part of how it's supposed to function and appear so that way when you interact with that chap bot which runs this underlying large language model um in this case GPT 3.5 unless you're paying for access to GPT uh 4.0 you get these sorts of ch

at interaction that you see down here now obviously a lot of other companies have taken notice and so you see a very similar sort of a thing happening but what's different so like Google for example has launched Bard and you can see here they have their own neural network which is going to be very similar in terms of like what's the kinds of mathematics that gets done with it and how it's structured but it's proprietary and they might have used a different slice of text to work with huge slice f

or sure and there um large language model um was called palm palm 2 for the particular version um that they've been most recently using and then the chatbot so being the car that runs on the palm 2 engine would be bar and so how Bard works then is okay well it should also appear like a chat window it should integrate with Google services specifically um one thing you may have noticed if you played with Google bard is that you can view multiple response options or the options all appear at once i

nstead of looking like it's being typed out there so those are some of the ways in which those chat Bots are programmed and structured to be different they both run on large language models although they may use different branded large language models um and this is all part of artificial intelligence now what makes this really confusing is that you know chat GPT has taken the spotlight it's it's gotten a lot of attention um but AI um includes a lot more than chat GPT and it's been here for quit

e a while um it includes lots of Technology IES that we've already used and gotten familiar with such as voice recognition um if you use an Alexa device or a Siri device at home image recognition which could be on the face ID to unlock your phones or to have um photos of your friends or family members autol labeled and detected um recommendations uh knowing sort of what would be good advertisements for particular customers or customer groups or other products that you might enjoy um because simi

lar people have bought those would also be a form of AI car navigation and finding directions have long been um an AI search problem um which also then ties into other search search is also Ai and that's been around for as long as we've been using Google as a verb um and language translation and and many other things so when we talk about AI um there's a lot more than chat GPT per se um but chat GPT and Bard and all of these other chatbot type Technologies and image generator techn Oles that can

sort of make um you know a fake image of Taylor Swift um at a Houston Texans game for instance rather than the Kansas City Chiefs game uh those are generative Ai and that that's the new genre that everyone's paying attention to right now because that can create new content it can create things that we've not seen before whereas these other things would detect existing patterns and sort of help us know which pattern um to detect in there so you know no one's going to be quizzed on this per se bu

t it's really confusing with all these letters coming out but AI is a bit different and bigger than chat GPT which is bigger um and relies upon things like a large language mod L all that does that help super helpful I I know that that was a little bit long but I said to Victor no you got like we have to do this it's super super helpful people do not just know that stuff so thank okay well let me tell you something I'm hearing a lot um especially in the Silicon Valley um and they're just saying

everyone is using chat GPT now everyone so Denise um is everyone using chat GPT that's like something you hear from your teenager right Mom everyone is going to the party I have to go to the party everyone's using chat GPT I have to use chat gbt well the answer is uh not so much so um challenge success as you know Dan did a very nice intro of us is a nonprofit affiliated with the school of education that I co-founded and we have um we were in the right position at the right time to do some data

collection and in the spring of 2023 which was pretty much chat GPT came out November uh 2022 so we were catching it just right at this past couple of months um we were asking a bunch of students and this is uh students at some different high schools if they knew what it was um and if they've used it we did some other stuff that I'm going to talk about in a minute too but um you can see right here I don't know what that is about 9% I have not used one 38% and I have used one 53% um then we decid

ed to go and collect more data in the fall of 2023 to say well maybe that was like a little bit on the early side and that's why they didn't know about it so much or you know what were the numbers like now and and now we've got a bigger n right about 10,000 student responses High School um I don't know what that is about 7% have not used one 54% have used one 39% and um so you could see like the answer is no not everybody's using it there's a good chunk about half that really aren't using it um

in fact a comparison to um some Pew data which is outside of Stanford from Fall 2023 32% hadn't even heard of it so you can see that even our responses with those particular high schools our particular Niche where a lot more students have heard of it whereas the national um sample from Pew a larger percentage hadn't even heard of it yeah so here's the thing it caused even though we have these numbers you'd have about half using it in this case 40% using it uh we're hearing that there's a cheatin

g crisis going on and we're hearing it in the media you might have heard um inklings of this school systems were Banning chat GPT because of the fear of cheating um so Victor fill us in what you know is this really a crisis should we be ringing our hands well I can appreciate that a lot of people are ringing their hands I do a lot of um work directly with teachers and superintendents and this is like the number one thing that they're asking about and I can completely understand so if you've spen

t time at chat GPT you can ask it to write a Sonet you can ask it to write a play you can ask it to write an essay comparing you know um the virtues of the Enlightenment and so that's that's uh quite impressive it's not the sort of thing that AI had been able to do in the past and we're seeing all these news stories and these are legitimate news stories about um AI having been used when it should not have been as particularly for these writing assignments and you can talk with various people and

there are certainly instances of people who have been using AI in situations where that is not really where we we expected AI to be used a lot of these when you read them are at the college level but you know given what we see is possible um and you had mentioned you know what is it the the way the teams say it so I'm GNA try to channel my inner teen using this popular meme we're kind of afraid that students are going to be very tempted to go off and uh favor chat GPT instead of actually doing

some of the writing that we're asking them to do as part of their their classroom assignments um but Denise I think that you can tell us a little bit about some of the additional work um that we've been doing thanks to challenge success being in the right place and at the right time yes so Victor and I paired up together were a pretty powerful Duo because we had all the access to schools and the surveys and we've been asking about academic Integrity along with well-being and belonging and engage

ment for years and so we were again at the right place at the right time paired up with Victor to then add some AI questions but we could still keep that same question about we had about 15 different behaviors that we ask about um copying from someone's paper with their knowledge copying without their knowledge um staying home on the day of a test to get more you know time to study and all these behaviors can be considered um different forms of academic Integrity uh uh violations depending on on

who you are and how you look at it so we had three high schools that we actually could compare and these are from the west and Midwest a private all girls high school a public high school and a Charter High School those two were co-ed and I'm just going to show you a little bit about what we found when we compared what those answers were on those academic Integrity questions pre chat GPT and then post chat GPT so you can look we decided to use 2019 only because I think you'll remember there's t

his little thing called Co that happened in 2020 and even 2021 people still still weren't quite back to the usual I mean some people would say we're still not back to usual um but we had good data from these three high schools from the academic Integrity questions on 2009 from 2019 and we compared it and you can see all three took it again in 2023 we also had two of them take it in 2022 so we had that data and we asked how often have you engaged in the following behaviors in the past month and w

e asked all those questions and if you look at sort of the overall percentages um there really wasn't that much of a difference so here's a good news bad news situation right the these are the percentages of cheating they're kind of high right and even the private high school the lowest we see is 60 up to over 80% in 2019 so there was a lot of cheating going on before anything called chat GPT was being used in schools um that should be a little bit of a red flag for people now if you fast forwar

d and look at 2022 and 2023 the numbers did not go up drastically they also did not go down drastically in some cases a little bit so we came to the conclusion that pretty much chat GPT did not open the floodgates to huge amounts of excess cheating based on what we'd seen in 2019 and and Victor you have even some more data to show on yes and you know just to remember it was November 2022 when chat GPT went viral so that we only have like the tail end of 2022 2023 has just ended well when you wer

e talking about cheating you were saying oh you know copying from someone else's test or um staying home and getting extra time and I had to say like that's not what my biggest concern is with AI or something like chat GPT or or chatbot technology it's that they're going to copy and paste online content that they generate um and so when when we looked at those specific behaviors um and this is just within one school but representative of others um in there it's also staying the same so it's turn

ing out that the students are not really that um compelled in large numbers to shift that particular cheating Behavior although there is definitely an interesting question about you know what can we do to promote a school culture of academic Integrity very at large and I know that this finding was was surprising and it was surprising enough because the New York Times contacted us um to share this and um I don't photograph well but we're really happy this work you we you look great come on you lo

ok great um but it has been getting uh getting word out there and we're actually seeing they looked at some other sources too and and we're seeing a pretty consistent pattern early on that it's still not um unleashing some new floodgates so um so that's been interesting but I'm not going to say and you know for sure there are going to be some people using chat GPT in ways that um we do not want to necessarily encourage in certain context um just as much as there going to be some people who are g

oing to be paying for a paper online that somebody else has written but it doesn't seem to be happening here at large but I do think that we are hearing and a lot of people are asking about you know AI it's raising a lot of ethical problems for schools I mean cheating is one thing but like there's a lot of things to think about here but I'm scared what what do we do what we need to be focused on Denise no it's a great question and and and think back to Victor's primer at the beginning of this so

we're talking about AI um not just chat GPT and its use in schools and some of these are old problems that we've seen just with technology in general and some are potentially new and the jury is still out so obviously the cheating particularly the plagiarism that we've been talking about um cyber bullying which has been around for a long long time now right when you start to um use AI some of the ways to detect um sort of the fake or who did it it's going to be harder right you hear a lot about

deep fakes um since you were talking about Taylor Swift I'll stick with the Taylor Swift um examples here Victor and you know it just came out last week that there's some really not great deep fakes going around about Taylor Swift um very hard to detect and also misinformation um who said what and especially in schools uh when these things can fly around very very quickly without necessarily the adults knowing about it until it's time to do damage control these are some pretty scary things the

other things to keep in mind would be privacy issues and data security um there's a lot of information being collected now um all in the name of potentially helping students right so you can track maybe who didn't turn turn in their homework the past 3 weeks and um automatically generate a note saying hey what's going on or whatever how much of that information being collected is actually secure how much of it do parents know are being collected there's a lot of privacy issues and data security

issues um that come up also I think if you've been reading uh you know that there's some intellectual property and copyright issues we have uh in the writer strike that that that that was a big sort of note of contention is who gets credit we wrote those words originally you're using our words to train this model and some kind of mishmash conf you know configuration gets spit out um is that right should we get copyright um rights there you know who owns that those new products that get created a

nd is it okay to use those products to train your language models and again that's I mean I think I think someone was meeting today in Washington to discuss some of this stuff the other thing that I think you should be aware of as an adult who has an interest in this topic are all the sort of algorithmic biases that we find because as Victor said it's trained on a lot of old stuff and a lot of old stuff already has bias in in terms of let's say gender or um social Moray right and social values b

uilt in but then also there's algorithmic biases just because the nature of the technology isn't there yet so it may not recognize different skin colors correctly it may um have some s of a kind of a bias when it comes to sort of gender and gender types um and then of course there's just the overall Equity issues about um using AI you know in order to use Ai and have it be a helper uh let's say at home for your homework you're going to need a device you're going to need uh connection right and u

m so we see Equity issues that abound and and uh there's a whole new field cropping up that Victor and I have been exploring called um technomoral virtues right because this opens up a bunch of things that again some we've been dealing with not well right in schools and now it's it's coming to be potentially new and even harder to detect so Denise what's really encouraging to me is that I know so many colleagues here at Stanford who are working exactly on these topics um and if we had more time

we' be able to have them come and share some of it but it's it's really exciting to be at Ground Zero for some of the progress there um but I know that you've heard um something like this what what was it again um that a lot of people were asking yeah so you know we one of the things we do at challenge exess we get we really Center the student experience we get students talking about these things and um you know students are saying well if AI can write our papers for us why do we need to learn h

ow to write like what's the purpose of School in the first place so um what are some of the changes right does AI change what we need to learn in school Victor like walk us through that okay I mean this is you know um a complicated question yes and no are going to have to be the the correct answers on this but I'm going to give one quick example here some people have played with chat GPT and know that it's notoriously bad at mathematics um there are plugins to work around it but you'll notice he

re I had asked it an arithmetic problem and it's very confident in its answer but if you actually look at the sequence of steps it gets very mixed up in there it starts changing what numbers it is talking about and this is a very easy sort of situation to reproduce so you know you could say that oh chat jpt is really good at doing all these things but it's going to be really important still that humans are able to spot check and catch these things so just as much and I think we've had this and w

e still have this debate with calculators do you need to learn math because there's calculators and everyone's going to have a phone in their pocket you know it's still is really good to know some of your arithmetic and mathematical processes because I punch in the wrong numbers all the time uh on my phone and to be able to catch that or to sort of have that sanity check associated with it so I don't think that we're going to be getting rid of some core content or the need to do that um but um w

e also need to be really careful about um knowing that when something can be done quickly and easily and come across confidently that it's not necessarily right when AI is involved um and that's actually a big concern I've got because you know we talk a lot about AI here at San frod in the Silicon Valley um but you know the the reality is and I keep seeing this as I travel to many different places um we need to understand AI better just as a whole um the Allen Institute did a large National surv

ey um not too long ago where they were checking on American AI literacy and this was questions like what does this device have artificial intelligence in it can artificial intelligence do this does artificial intelligence do that and only 16% of adults could pass what would be basic AI literacy tests um so like knowing for example my remote control does not have artificial intelligence um built into it and you know we one of the things that we look to in this country is schools being one key pla

ce to make sure that we give comparable and equal footing for students to know what they need to thrive in society and economically um but it's I have to say this schools are having a lot of challenges to face right now and I am so greatly appreciative to our Educators and their administrators um who are working really hard and you know if you look at all the curricula and stuff and all the classes and especially coming out of remote learning um from the pandemic there is no room in the school d

ay there is no time nor capacity to say okay well guess what we have ai so we now need to learn AI on top of everything um and even if we did that you know so much has changed just in the last year just even in the last few months and so if teachers are going to be up to speed and teaching this um this is going to be a really hard problem because everything is going to keep changing the rug is going to keep getting pulled out from under them and so we need to come up with some sort of new approa

ch or new solution and that's one of the things that I've been really excited that I've had a number of people here at Stanford including students especially who've been so excited to build out curricular resources for teachers that we because we are at Stanford where we have a leading School of Education worldwide reputation and and important findings plus the same could be said for computer science and artificial intelligence and we are just right near each other that we can be in dialogue and

start building out here's a quick lesson or um material or um some instructional uh resources that a teacher can use to talk about Ai and the newest developments about AI because a lot of them are crossing through our campus in one way or another and we also do this by way of codesign so you know we want to make sure and this is what we know as as a school of education that the best way to make things work um for for schools and teachers is to really understand and work with um their circumstan

ces and needs and give them the things and the forms that are going to be nudging in the really right directions um and so we've been building out and are continuing to build out and I have this line of students which is super exciting who want to be involved in and make a difference within education although that may or may not be their long-term career trajectory but we're giving them this whole development internship experience in building out curricular resources and getting to know and work

closely with teachers so this will be one of our endeavors it's uh yet another acronym on the Stanford campus but craft. stanford.edu and and we are continuously adding and it's been wonderfully supported by the school of education um and de accelerated for learning as well as a number of other units um on campus and you're doing some stuff also a chall success right yes we are and I should say too I'm super excited about craft because my curriculum construction students this quarter um are act

ually working with vict and his team to design some curriculum uh around Ai and AIU so that's you know Victor and I we're it's been really fun to partner with Victor in ways that we haven't really done since AI came around so AI brought us together Victor which I'm very excited about um but yes at challenge success so Victor's got his team working on curriculum and a whole bunch of other stuff which we're going to talk about in a second but at challenge success folks have been coming to us and s

aying look first of all we get that cheating's been a problem for year can you help us and now that AI has come in can you also help us think about not just classroom based policies and things that we can do to mitigate that but also what can we do in terms of schoolwide policies so at challenge success we partner with schools and work on both classroom policies and practices and schoolwide um policies and practices and our little sort of Trifecta our theory of change um from the research is tha

t wellbeing belonging and engagement all go together and that's what really leads to thriving students so if you look at sort of why kids are cheating in the first place a lot of times it's not because they they are lacking the morals right A lot of times they are exhausted they're doing homework super late at night after lots of a long day of school and extracurricular activities or work or taking care of siblings Etc um or they really don't understand the purpose behind it and it kind of looks

like busy work and they'd rather put their time then on things that they don't think is busy work or they don't feel like they really belong in the first place in that cassum when they don't have a relationship with peers or the adults uh in the school or the classroom that would make them feel like they don't want to let those people down or they want you know we wouldn't cheat because we really want to learn and learn from this person so when you it turns out when you focus on student well-be

ing Health Sleep all of that belonging where they feel feel like they can bring their whole s to class and someone has their back an engagement where they see the meaning behind what they're learning and they're excited to do it cheating goes down um and some of those other issues that we've seen around cyber bullying Etc also go down so um if you're interested in more um I know someone just put the website in the chat but we'll also have it on the last slide as well we a lot of stuff but I thin

k that there's one thing that we keep hearing a lot of that maybe you could surface yes so I think and it's happening today in DC right now right folks from meta and Discord are like on the stands right now if you read the New York Times this morning the future of AI is ultimately going to be defined by big Tech like what is our role here aren't we powerless to these big tech companies and Victor you have a a very optimistic and positive answer to this I I I think it's well founded optimism I me

an we are in such a unique position both metaphorically and physically I mean physically we are in the Silicon Valley and a lot of the companies and the employees that they bring on and the new startups are coming out of Stanford so they are in our classes and they're participating in these conversations and um having some of these discussions and these provocations and learning that along with the Technologies and the ethics are associated with it so on the one hand we are helping to feed the N

ext Generation that is going to be shaping this technology but what's also really cool position wise is that you know like I said we have such a phenomenal School of Education such a phenomenal uh Department of computer science we have a phenomenal Institute for human centered artificial intelligence and we've been able to through the Stanford accelerator for Learning and at The Graduate School of Education really bring that all together and we've launched this initiative where we've had tremend

ous interest um on new seed grants which are there to create entirely new possibilities start up new research studies and projects to speak to the moment and I would not have enough time to talk about as many incredibly cool things that are coming from you know first year students all the way up to senior Professor with tons of awards and lots of prizes but just to name a few you know how could we use generative AI to make um new lesson plans and ease into the amount of time teachers are having

to put into their preparations and th thus free them up to engage with students more and sort of address those concerns that you had talked about we um I spoke about um uh voice recognition um and so we have some really amazing uh projects where it's taking um audio from classrooms and giving feedback to teachers to let them know are you creating enough speaking opportunities or asking really productive questions on some of the content in there um we are looking into ways that this can be integr

ated into medical education or merge with virtual reality to create really realistic helpful agents and to actually think about what form of conversational partner using what sort of soci or dialect um is going to be most relatable because like I said a lot of the technology has been built on what happens to already be on the internet and that doesn't necessarily represent everyone's voices on there we have whole new imaginings of what English class in the future could look like in partnership w

ith teachers and ways that you could use AI as like a debate partner or an idea generator um to help to support writing but not replace writing so it's it's really an amazing time here and I think that Stanford because of the platform because of the capacity because of the people we have coming through you know big Tech is going to be very visible but we are going to be continuously shaping what big Tech and what policy makers are doing um I mean I just I literally just flew back from DC late la

st night and this is you know something that our colleagues are continuously doing so I think that it's not strictly in the hands of big Tech they will be a big player but Stanford is all Al a massive player in all of the space too I love that that makes me so happy Victor thank you um so if you want to check out uh more about craft or challenge success there's our websites but I think it is time now for us to turn it over to Nate for some Q&A it looks like we have quite a few questions too so I

'll stop hearing then uh yeah make sure everyone we can put those in the in the chat as well uh Nate hi good to see you back thank you both so much for that great presentation um and uh yes there are indeed a ton of questions coming in over uh in in Q&A a lot of them really fascinating we appreciate all of your insights but this is the this is the particularly good stuff so um uh I'm going to do my best to group some of the themes because they're they're they're they span lots of different subje

cts um but I think one that uh that started off as just sort of an interesting like I think segue from some of the things that you've said is the using example of when po pocket calculators became ubiquitous in classrooms um and uh you know uh the lens the Contemporary lens of uh of of Education goals pedagogy affects how people uh look at these new tools and I'm wondering with that comparison in mind and you've already alluded to this a little bit um you've talked about the potential that AI br

ings as an educational tool I'm wondering if you can expand that a little bit more using that sort of specific example and a as a background so certainly I mean you know I think we'll be very um limited if we only look at what we've done in the past just as Dan had said in his introduction and just try to reproduce that as we develop new technologies we come up with new possibilities and um whole new things that we haven't necessarily imagined for positive and and negative and so that means that

you know we are going to be looking at very powerful things that we can do with artificial intelligence that we had not been able to do prior to that and that is going to to be part of you know the educational ecosystem um there will be some persistent challenges that are going to be hard problems that we will hopefully get more purchase on but should we start to um think that there are some things we can soften up on because that is not something that is going to be the best use of human power

but would actually be great with humans aided by technology absolutely and that's something that we're going to continue to work on modeling um and getting the message out there but also being responsive that what is at the core of this we want to make sure that the human relationships can still be maintained and some of the things that we know are really hard problems that really humans need to be involved in um are continuing to get that attention so you know just like calculators are here we

still have math class um are there things that we could explore and broaden for sure and are we having work done to push that envelope absolutely same thing is can be happening with Computing in AI I wonder there's a whole family of questions that's great thank you Victor there the whole family of questions about um uh not so there's the the what what what should we be worried about and try to try to you know protect against on the AI side of things but there's also a set of questions and parti

cularly from clearly some Educators who are uh joining us online around how AI in its contemporary form can actually assist teachers um and in in insist instruction and even in assist some of the the the big three uh elements that Denise you were talking about in terms of of belonging and engagement I wonder if you could Riff on that a little bit the the the way AI is a a potential uh right now a potential uh tool to assist teachers and Educators in the workplace yeah I'll speak really quickly a

nd then pass Denise so I mean one thing I'm excited about is I do see more movement this year within the tech industry at tech industry to be much more attentive to what can we do to help teachers rather than how can we um exclusively only focus on a tutoring platform which will be very interesting development but you know teachers are juggling huge amounts of information and needing to produce huge amounts of information so to streamline that um and to make it so that way less of their time is

doing the busy work of you know digesting a bunch of information um and writing comments that are going to sound fairly repetitive and being able to have more tools to support that or to give good recommendations as additional quote eyes and ears of what's happening in the classroom is going to be huge and it should not be taken as licensed to um say oh this is going to just make it so we can increase the number of students teachers are working with no it's going to actually let teachers do the

work that brought them into the teaching profession again um but I think that being able to cut into a lot of that bureaucratic part of teaching that takes them away um and then help be more useful input or more useful resources such as I need a sample written response of this sort I need a sample assessment like this um is going to be really powerful um for them and also help to give ideas on how to engage with students with whom they're still learning how to best connect with them and might kn

ow like you know maybe Taylor Swift is a middle-aged phenomenon and there's someone more popular and they then AI can maybe help connect some examples to get us started yeah and I would just add thank you Victor and I think um again that human component is not going to go away you still need the human to check on all of these things so we I I've heard teachers who you know uh let's just take writing letters of college recommendation if you're an English teacher or a math teacher um and you teach

seniors you may have you know 30 to 50 asss um a year and that takes a lot of time AI can actually really change that by pulling in some of the very concrete things you know about each individual student and individualizing those letters but you still as a human have to read over those letters and make sure it doesn't sound like a robot wrote it and make sure that there's no hallucinations which is what's called when AI is making mistakes right so there's all sorts of efficiency uh work that I

think teachers are very excited about um in the world of mental health and well-being there's a lot uh as well there's you know even just the basic reminders of have you had enough sleep or is it's you know it's time to turn off the machine and go to bed right kinds of things all the way to um I was just talking with the company that they happen to track students emotions on any given day uh the student weighs in they can then through AI generate a letter home basically saying hey every Tuesday

night you know Johnny is feeling really anxious and scared about getting his homework done what's going on on Tuesday nights right so there's some really interesting things coming from this I will want to add though Denise because you mentioned wck letters and I know that there's uh I've heard a lot of teachers have a very strong response because there's something very personal that we want to express and there's a lot of interpersonal Communications that we want to make sure are expressed as hu

mans but you know just for the recck letters I write Gathering up all the materials getting reminders of oh remember this one activity the student had done or some of these other things and if those could come up as suggestions to make my writing easier and I can still really very much use my voice that would be a very nice Aid and Ai and help me to stay a lot more organized because I want to make sure I can say what I can but I spend so time digging up the old assignments checking through all t

hese different notes that i' had made and and putting it all together whereas some of that can come up as information as a reminder as I'm in the word processor for instance absolutely great uh thank you actually staying in that train of thought and maybe this I'll start with Denise um what do you say to people who say you know why train teachers at all you know this is can't can't students eventually just have an AI tutor um uh who tells them everything teaches them everything they need uh uh a

nd personalizes their learning in uh all the right ways quote unquote what do you say to those people well it's a great question I mean if you go back to sort of that Trifecta of well-being belonging and engagement and you think back to you know um uh the the maslov's hierarchy of needs right we need to feel safe and we need to feel secure and we need to feel in relationship before we are ready for what he called Enlightenment right before we're ready to learn and so um as it gets better I know

that people have think that they're in a relationship with an AI uh they've been doing this with sort of elderly people to combat loneliness but there is something about actually having a human right who you are working with who can make those calls and see the limitations of the technology and see a little bit more of how to really bring it to its full potential um it's it's the aid right it's like that old line about it's not that doctors are going to be replaced Radiologists aren't going to b

e replaced because of AI it's that Radiologists who don't use AI are going to be replaced by Radiologists who do use AI it's going to make it even better but you can't lose that human component both in terms of just literally right now checking right um that things are done well I in my curriculum class we can spit into chat GPT you know pull out a curriculum on X but but there is knowledge there right there's a critical eye needed to make sure that that is actually right and right for our parti

cular students what I would also add to that is um you know there's so many people who and I applaud them who have changed careers and gone into teaching and you know their jaws just go slack they're like this is so hard this is one of the hardest human activities and tasks and AI is a great Aid that can sort of imitate some of it but there's so much that we can't even fully articulate well um and so much that I I think that just as much as we would say oh we don't need people to be parents to t

heir kids or we don't need to hang out with actual friends there's things that are going to intersect both with values but also I think that intersects with a complexity of human experience that for Generations we know intrinsically the importance and the messiness of it and while we can have things that will help that's not going to be the same or even worthy pursuit to replace um it will just broaden broaden some of those options that are available yeah I I it's everything that you say you bot

h say resonates as somebody who works in education with me but also just as a parent I have teenage kids and the trifecta that Denise talks about and the Dynamics you were just outlining Victor are things that we want to lean into the personal connection and all of those kinds of things and to the extent that those things can be made made better by AI in some way because of efficiencies or tools that teachers don't have it's that's the very sort of optimistic side and positive side but as parent

s we think about that a lot and I'm sure people on the call do too okay I'm going to switch gears a little bit um and talk about higher education there are a lot a cluster of questions about the impact of AI on College and there's a lot of conversation nationally about higher education in this country um and I'm just wondering if you have General thoughts about what you're seeing in Trends right now and since this is a Stanford program I think people are particularly interested in what Stanford

students are experiencing and and what we're experiencing with them on campus related to Ai and I guess the first part is the sort of more pessimistic part do we see negative trends that we're trying to protect against and then the second part is what do we see that uh is showing up today on campus with our college students that um gives us excitement and hope those are fantastic questions and I mean I think there there's a huge conversation to be had about higher education today and right now a

nd I mean there's a lot of conversations and I think teaching moments about U matters like academic Integrity or plagiarism or being comfortable having dialogue with somebody and seeing or taking positions that may maybe um you're not feeling confident in in expressing and so to the extent that we stay focused on what those challenges are and how we can Aid whether it's sort of virtual ual conversation Partners or seeing how instructive certain cases are and like discussing where is my unique co

ntribution to something and where is the tool um you know something that we can very much broadly accept as as appropriate in there what I'm seeing with students and I think some of this is is in part because of Stanford's entrepreneurship um orientation is that there's a lot of students who want to try to create something new and you know credit to this generation knew that will address long-standing problems in the world that have been kind of passed down to them they're very concerned about c

limate they're very concerned about um you know people feeling included they're very concerned about matters of equity and opportunity and so I hear these fantastic ideas and real motivation from students to participate in workshops and classes in competitions to develop those skills I do also hear some students are very nervous or it's like well I came here to major in computer science but if that thing can write the program do I need to well just as much as I showed that math example it can pu

t out code it doesn't mean it's good code it doesn't mean that it's right code um and coding languages change and you know some of our most sophisticated techniques will continue to change and that's a very hum driven process so I think there's still going to be great importance um being placed on that at the same time you know it's been a year and change um and we've been one of the fastest groups out with some of the high School research being able to look free and post but you know with highe

r education like there's a lot of questions that we need to examine and there's a very different set of circumstances so I would say this is where research like what we try to encourage here at Stanford and through a school of education that is leading the way on research um is going to be continuing to do the work to find out what is happening um beyond what we just do an initial glance online for yeah and and I would say too it's the same things that you would do at a K12 school to mitigate ch

eating that work also at the higher ed level with some caveats so uh if the students are completely overwhelmed if they don't see the purpose in the assignment if they don't feel relationship with with uh you know the adult um yeah you're going to see some more academic Integrity issues um the way that I do it in my class and I think we've talked about this as a school that as a whole is um I allow chat GP use on assignments but it has to be cited that they used it they need to tell me sort of w

here and when they used it um because I do see it as a tool and I also make assignments that build in that they have to use their own personal experience as part of the evidence behind the claim um which again you know it's not going to be there's ways to get around that um but that that is a is a preventative model and then I have students who will you know in the middle of a discussion in class plug into chat GPT to get some discussion points and I'll just say well what do you think about that

right we know what chat GPT now says about it but what do you think about that so you have to be uh skilled enough and and really understand the students needs to do it well but um yeah the same same we're seeing the same thing happening K12 at higher ed the difference is higher ed you might have a hundred students in a in a lecture hall right as opposed to that K12 teacher who may really know the students and kind of know hey this isn't their voice right so I think Det is a little bit trickier

Victor and I have talked about that before great and and just one more higher ed question is there any impact on the exercise of getting into college in other words uh essays and you know all of those kinds of entrance requirements do we do you see any Trends there or anything to be paying attention to we're really fortunate to have some of the leading Scholars on higher education who've been engaging on this particular topic um I mean I think right now especially this year there's a lot of new

thinking about how to be most effective in the admissions process um and what sorts of inputs to take in and so you know the solutions are yet to be found but it is uh very much on everyone's awareness and there's a real commitment to pursuing responsible um Solutions on that front um and so you know stay tuned I think that we are going to make Headway on that um but for everyone who's been following the news a lot has been going on chat GPD is just one of the things that are in there you know

I think it comes down to sort of us as parents caregivers and Educators to also say look are you as a student as an individual as somebody's be participating in society what do you want to uphold to yourself what are you trying to accomplish here and where and how will you model what what you envision for your own future and and for those who are coming after you and provideing support and encouragement for that to be um what they exemplify great um okay I I we're almost at time uh before I turn

it back over to Megan here there's so many good questions I there's no way I can ask all of them um I do want to give a quick shout out to uh what is clearly some there's a bunch of high school students on this uh participating in this program today and they are asking great questions and I'm just happy to have them here uh a lot of enthusiasm also thoughtful questions about uh impacting their experiences as they live them right now including an anecdote about what what about my my my class eve

rybody's asked my teachers are requiring everything in shorthand and we've never had to write so much by hand in our Liv in our lives Denise do you have any quick observation on that before I turn over to going back to everything in shorthand is is not going to be a a sustainable solution and also really hurts the people's wrists um never mind a whole bunch of folks out there who for whom that's not going to work for the different ways that they're bring work so no that's not the answer um uh ha

ve them check out challenge success we put on workshops have their teacher come to a challenge success workshop on how to use uh AI in a way that that you don't have to go all the way back to the shorthand perfect good that's a great segue check out challenge success um I'm going to turn it over to Megan to do the the closure and the and the thank you but just from our perspective thank you very much for your time and Megan take it away great have to say I'm a big cursive right advocate so so go

od afternoon everyone my name is Megan squeezy Fogerty I'm honored to serve as Stanford senior associate VP for Community engagement and join you to close today's program so first and foremost thank you to Victor Lee Denise Pope and Dean Dan Schwarz who are among so many researchers at Stamford who focus on the challenges and opportunities we Face together in our region and world the recording of today's program will be available on the continuing studies website as well as Stamford YouTube YouT

ube so you can watch for that in the next couple weeks we also hope you'll stay involved with Stamford you can learn about events open to the public many of which are virtual at events. stanford.edu and if you live closer to campus the recent rains have turned the Foothills green and we invite you to enjoy the beauty of the Stamford campus by taking a walk at the Stamford dish or viewing art on campus and in our free museums I also encourage you to check out Stamford Athletics we know women's ba

sketball is exciting and many winter sports are in full swing so thank you for participating in discover Stamford for you please let us know your thoughts on today's event when you get the survey and if we can provide any information you can reach out to me and Stanford University through community. stanford.edu also at that website you'll find a list of our favorite Stamford podcasts under the Discover tab including the graduate school of education schools in podcast which features fantastic di

scussions led by Dean Schwarz and Denise Pope we look forward to connecting further as we all work towards a better world stay in touch and have a wonderful day thank you