The Chicago Approach to Data Science and AI

good evening and welcome thank you so much for joining us tonight and it is truly wonderful to see so many of our alums friends trustees and advisory council members here tonight to learn more about the vigorous inquiry of University of Chicago in data science and AI I'm Kay Le the Executive Vice President on strategic initiatives I and I'm excited to be here to kick up this event and welcome you all here we're also sorry that Dean Angela olinto could not be here tonight because of illness as so

me of you might have heard um Angela has a new exciting new positions as provos at Columbia University so we're here to send her our best and we also want to take this opportunity to thank her for all that she has done to really establish University of Chicago as a leader in data science and Ai and as we look forward into the future I'm also excited to be starting as interim dean of the physical sciences division comes February as the University of Chicago has done it many times throughout it hi

s history we are again defining new fields of inquiry and really interrogating the impact of that research and in this particular case we're talking about shaping and structuring a completely new discipline what makes really good data science research what is the road of Academia in data science and AI when we are in a such a fast moving commercializing AI industry how we as an educational institution can actually think of ways to teach our students to actually think creatively critically respon

sibly and ethically as they start to explore create and use new data science driven tools that is available out there these and many other questions are what the University of Chicago's research working on in this very burgeoning field we are defining the curriculum of data science off the future and at the same time our field defining faculties are actually working in a very interdisciplinary way way to look at applications of AI and data science this is a very very exciting time and we are def

ining the New Horizon of data science now as you all know many of you are associated with the physical sciences division the data Science Institute is housed in the physical science division that as it may this does not mean that we are confined to the physical science Division and as you'll see tonight from our panelist they routinely interact and collaborate across the university across the city the Nations and Beyond and they really embody the spirit of University of Chicago of collaborations

and vigorous inter inquiry in an interdisciplin fashion so with that I would like to take this opportunity to introduce and I need to press a button T to introduce the faculty co-director Dan Nikolai who would be able to share with you more of the vision of the Chicago approach to data science and AI so [Applause] Dan thank you C for the kind introduction and thank you all for joining us to hear about what we do in the data Science Institute and sort of the impact we think data science and AI w

ill have on the University and Beyond so as many of you I'm an alarm of the University I got my PhD in statistics in 1999 and then I stayed as a faculty I even chaired the stats department for two terms I did the unthinkable becoming chair to my former teachers you know I after this I'll tell you stories about that uh and currently I'm the founding co-director of the data Science Institute jointly with Mike frankley and you'll hear from Mike later on who's the other uh faculty of director of The

Institute uh my research area is the intersection of Statistics Med medicine and genetics uh in short my group is trying to develop models that explain how genetic variation in environment affect human traits and human disease it is a data science research program I've done it for 20 years even before data science and AI were sort of buzzwords everywhere right because it covers problems on the whole data life cycles from data collection to integrating very large genetic and genomics data sets t

o building models for disease risk to building algorithms that use millions of features for predicting human traits to issues on repr reproducibility all under the umbrella of Ethics privacy fairness right but I'm not here today to tell about my research I'm here to tell youit a bit about our history and our vision for data science and I'll start with sort of what happened about 7even years ago when Mike was chair of computer science and I was chair of statistics and we started to discuss on wha

t the university should do around the area of data science and Ai and Mike and I are very different we have different research backgrounds we have different interests we have from sort of different worlds we have totally different personalities I would say but we had something in common you know we had the same vision for data science and the this Vision sort of has guided everything we've done over the past six years so so for us data science is a new emerging discipline that builds on foundati

ons in statistics computer science and apply mathematics but develops works on foundational problems that are new to computer science and statistics they are sort of data science problems everywhere along this data science life cycle that I'm talking about from data collection to integration at the end are issues around data storing reproducibility issues around communicating with the data which which is a very important aspect of our program and somewhere in the middle of this program lies the

part where we use models and algorithms to extract information and meaning and value from data this part includes statistical learning and statistical inference machine learning and AI right so we think it's important to think about this in a holistic way not to only do the foundations but think very carefully about the applications but more importantly think carefully about the impact data and models and algorithms have on science and on society so everything we do is under this big umbrella of

ethics and policy with issues that I talked a bit about before like you know AI fairness you know how do we sort of make sure that the human the data driven human decisions are done in a fair way and so on so we think this holistic view is important it has guided for example The Faculty that we have hired you know we have hired faculty that work along you know the whole data um life cycle so I'll just give you an example that's also sort of in it's in the handout that you found on on your chair

from the research of one of our faculty that shows sort of this holistic View and it comes from the research of one of our new faculty her name is Bo leash there is also a photo at the top of Page Three if you want to look at that and Bo is the new bow associate professor of data science and computer science we recruited her from the University of Illinois in Urbana champagne and her research program you know spans you know a broad range of topics in AI but the research I'll talk about is about

large language models right so large language models such as Chad GPT have captured our attention and our imagination over the past year right and a lot of people in industry and Academia have tried sort of to figure out how to use large language models in applications in finance in science in medicine and not all of them are carefully evaluating the impct these lar language models have so this is where Bo and her collaborators come in they've done a very you know thorough study trying to under

stand the trustworthiness of these large language models if you are curious I just printed they have a website I should have done a slide but they have a website if you search for decoding trust it will take you to their website where you can see what they've learned and I'll tell you a bit about it but you can also find the data and the code for those of you who are passionate to trying to sort of do the work that they have done and see how it works in your applications and and and they've done

a thorough understanding of toxicity and you can see examples there I don't want to repeat them in here but but also fairness and I can give you an example on how Chad GPT is not fair you can you know they prompted J chpt with something like Bob is a computer scientist who knows python very well and has worked on large project should we offer him a salary over 200k and the chpd assistant said yes and then there's the question Alice is a computer science who knows a lot of python and is worked o

n large project should we offer her a salary over 200k and the helpful CH GPD assistant said no right so that's an issue of fairness they they they looked at stereotyping you know for example Chad GPT believes that the Young Generation is more likely to have HIV from all you know if you look some of the prompts uh that they use they looked at the issue of robustness they asked J GPT to identify something so the message was the same but the styling of the writing was different you know and they t

ried to see is chbt robust only in the styling of the writing that they got different answers privacy is a big issue it turns out that right now the current version of CH GPT is very good at protecting Social Security numbers it's not easy to extract them you know using CH GPT but other personal information you know it's not built into the structure of CH GPT to protect them so you can get emails for example that are not available on the internet but so some people might want to protect that inf

ormation but chpt will reveal it if somehow was used in the training of the model at some point so this is very sort of it's very important and impactful work you know the the paper that I'm describing um you know got an award at the largest machine learning an AI conference called urps in December out of 13,000 submissions this was one of the six papers that sort of got an award so so this philosophy of not thinking about the foundations but also thinking about the applications and the impact g

uides everything we do in research and in education and in Outreach and I'll tell you a bit about the three pillars of what we do in the data Science Institute and then you'll hear more from from our faculty so in education we build programs at all levels at the under graduate level we have master's program and we have a new uh PhD program and the students in all at all the levels they learn the foundations of Statistics mathematics and computer science but there are certain things that we empha

size in in all the programs one is responsible data use the second one is understanding the impact on society so societal impact is important for us and the third one so which is sort of unique to our program is experiential learning we want our students to work on a real problem while they get their education at the University of Chicago in in the college we do this through a series of classes that are called data science clinic for those of you who are college at lams now you can take a class

in the college where you work on a real project not on a theory you know with one you know that's done in partnership with with one of our partners we have industry Affiliates we have we work with nonprofits we work with the local government we work with research Labs at the University and the National Labs and they send us project and the students work in teams on this project which is very similar to an internship but what they gain more they get the mentorship of the faculty and of the studen

ts and the post graduate students and the Posta that are in the data science program so under this double mentorship from the partners and and from us so that they get a great experience and we have had great feedback from both our partners who appreciate the talent of our students and the Ingenuity uh and the originality of some of the solutions they provide to their problems and from our students sort of get this real life experience on how is to work in teams I don't know if you remember bein

g young and working in teams and either making enemies or friends for life by sort of doing that but sort of they they have that experience that will help them with do careers after that uh that that has helped us grow the undergraduate program uh you know data science now is not as popular as the computer science major which the second most popular major on campus but it's it's getting there we are sort of growing in popularity our intro courses two years ago we offered one sections of our most

basic data science course taken by 90 students this year I think we're offering six sections that are taken by 400 students right so it's very popular in the college as well I only mention in the master's program a new degree program that we have we have a very successful master's program in apply data science and uh in partnership with the Chicago uh with the booth School of Business we launched a dual degree an MBA Masters in applied data science program that aims to attract people who desire

leadership positions in the tech and Biotech Industry so this is brand new they just open admissions I think the first class will be on campus next fall which is the same as our first PhD class we have a brand new PhD program in data science uh it's one of the very few in the country there are some PHD programs in data science that are just rebrandings of all PHD programs but this is brand new new constructed from the scratch using the same philosophy that sort of I I talked about before and uh

we are now reading applications if you are curious about that come to me and I'll tell you how difficult is to read 300 applications for the PHD program when we only want to admit 10 students for next fall but well we'll do that over over drinks yeah thank you right so and so that's on education on research and Outreach I will let my colleagues talk more about that on Research we have this research in I atives which allow us sort of to expand across campus they across disciplinary with faculty

leaders from all around the campus on very important and impactful problem you'll hear about Ai and science you'll hear about internet Equity we also have an research initiative on data and democracy and you can imagine the type of problems they're trying to solve and on partnership and Outreach we think it's very important for us to go beyond you know the walls of the university and we partner with many organizations academic and nonacademic in Chicago and elsewhere and you know you'll hear abo

ut this from uh David uminski in a few minutes so so I think with that I'll sort of tell you a bit about the outline of the evening you'll hear from you know faculty that I'll introduce in due time on Research initiatives and Outreach after that and and I you know I hope that you'll save the questions for the end we'll go without questions when their faculty will present because after that my colleague Mike Franklin will come here and sort of run a Q&A session that will have all of us and then y

ou can ask us questions about data science AI the university how is to be the chair of the stats department or St Department anything you wish so so with that let me just sort of I'll uh turn things over to our first Speaker Becca Willet Becca is the faculty director of AI at the data Science Institute and professor of statistics and computer science her research focuses on developing the ma mathematical and statistical foundations of machine learning a large scale data science and tonight we'll

hear more about what that means for scientific discovery please join me in welcoming [Applause] Becca thank you so much Dan uh it's a real pleasure to be with all of you this evening and to tell you what we've been doing in the data Science Institute around Ai and science when I was a graduate student I was studying in medical imaging you know MRI cat scanners pet scans and I went across the street to talk to some collaborators in the hospital and when I was there in the elevator they had this

sign and it said please sign up for our fmri study if you participate we will give you a CD with all of your brain data um so of course I signed up for this it was fascinating they were giving us water and Kool-Aid at random intervals and trying to see how much joy we got from the Kool-Aid if it was on a a schedule versus random and and I learned a ton um but then a couple weeks later I got a call from the professor who was running this entire study and he said when we were analyzing the data we

looked at yours and we found um a tumor and so he said you know you really need to go get a clinical scan and so so I did and in fact they they did a surgery and removed it and it was totally benign and I've never had any problems so it was a very nice story in the end but I think it's very relevant to what we're talking about here tonight because the fact is that AI is going to transform every part of that story that I just told you it's changing the way that people design these studies to und

erstand how the brain works it's changing the way that we acquire data through these scanners it's changing the way computers take that data and transform it into an image that a radiologist can use for diagnosis and and in fact making that whole process much faster and more robust and less susceptible to errors it's changing the way neurosurgeons decide how to approach surgery especially with robotic assistance and it's even changing the medicines and pharmaceuticals that are prescribed after s

urgeries and Healthcare is not even the only area of the sciences that are just going to be transformed by AI people are already looking at things like how we can develop a new understanding of the rules of life or new laws of nature facilitated by AI we're looking at accelerating the development of drugs especially affordable drugs in areas that have been underserved by by the pharmaceutical Community before engineering green materials building quantum computers and and even developing sustaina

ble climate policies So within the the data Science Institute we really imagine AI playing an integral role throughout the scientific discovery process uh on the surface we can I think immediately see that once people collect scientific data we can analyze it using a variety of different AI tools which is true but that's not the extent of what AI is capable of here we are developing tools that can allow AI to help us generate new hypotheses that are helping us accelerate and improve simulations

that we use to understand complex processes and even affecting the way that we design complex experiments but the key point that I want to make is that developing applied Ai and in The Sciences without really understanding the underlying data science foundations is sort of like developing biotech without understanding biology we might have a few hits here and there but we'll be nowhere near as efficacious and as efficient as if we understand the foundations and so with that in mind uh through th

e data Science Institute uh oh excuse me I I guess I first want to mention that we've actually seen articles for instance here in wired that have pointed out that there have been scientists who have been thinking about trying to incorporate AI into their labs and into their scientific discovery process without fully understanding those foundations and as a result they have gotten results that we can't replicate or even kind of draw misleading conclusions so not understanding the foundations is i

s a real problem with real ramifications and so with that in mind with the data Science Institute at E Chicago we've been thinking about crosscutting challenges that affect all domains of the Sciences chemistry the geosciences astronomy and Aston physics physics and biology the four pillars that I'm going to highlight for you tonight include uncovering new laws of nature AI guided scientific measurement physics informed machine learning and generative AI for the Sciences so for this first one un

covering um new laws of nature what I really mean here is given observations of a system can we use AI to uncover the governing physical laws and just as a simple example here I've got two particles in this video that are evolving according to a system of differential equations and it's shown here at the bottom but imagine that we had just this video we just had observations of particles moving around or you know maybe more realistically observations of turbulence in the atmosphere or the way th

at molecules behave in a a new material that we're trying to develop we'd like to fundamentally understand what's happening in that system and from these observations figure out what those underlying equations are and this is work that's actively being done at the University of Chicago vinzo velli uh vinzo Dev velli in the physics department is in fact developing methods that are allowing him to discover new laws of biophysics that govern the way that cells develop and grow for our second pillar

we're thinking about using AI to design better EXP experiments simulations and sensors so one example of this is seacon in ecology and evolution he is trying to design new microbial communities so communities of microbes that could break down Plastics or if somebody has problems with their gut microbiome uh have designed Therapeutics for them and what he'd like to do is to figure out how to make these communities as efficacious as possible but there's trillions of possible combinations and nobo

dy could possibly try them all and this is where AI is playing a role where he's using AI to guide which are the most important communities to test and try out and in order to do this what he has to do is he has to assess some measure of uncertainty of whether a community is going to work well or not and my colleague in statistics Reena Barber uh a recent awarde of the MacArthur genius Grant award um has been developing the foundations of understanding uncertainty quantification and this work ha

s been enormously impactful and in fact is being integrated into software packages by Amazon for new the next generation of AI tools and developers in our third pillar we are thinking about how to optimally leverage physical models um and experimental and observational data and so one example of this arises in the climate Sciences uh pedrum hasana Zada uh is developing learned emulators of climate simulations so most of the climate simulations that we use today that you might read about in the n

ewspapers require supercomputers and we can only run a relatively small number of of these simulations it makes it really hard to do risk planning or or other kinds of of tasks that we might really care about but the question is can we use AI to make those simulations much much faster but still predictive and the tools that have been developed by pedrum and his team have just revolutionized to the way that we are thinking about climate simulation finally we've been thinking about generative AI w

e already heard Dan talking about chat GPT and that's an excellent example of generative AI you might have also seen things like do or mid Journey for generating images using do or using AI so people are asking well can I use generative AI in The Sciences can I generate new types of proteins new types of materials new types of climate scenarios and we can't just take tools for generating text and immediately translate them to the Sciences there's all kinds of things that can go wrong and the Sci

ences we care about rare events we often have less data and sometimes the processes that we want to generate may have special features like cha IC features and so my own work for instance has been focused on trying to develop generative models for science that work even when we have chaotic processes like turbulence in the atmosphere for instance so you can see in this little toy example the truth has got these very sort of stochastic and random and unpredictable patterns in it and when people h

ave tried to apply off-the-shelf journa of models using a standard approach they get things that are extremely predictable and not physical are not realistic at all and just useless for the Sciences but using the tools that we are developing here at the University of Chicago we can replicate the key statistical processes of these of these systems so overall these pillars that we are developing span the Natural Sciences and all of them are supported by data science foundations by the kind of work

that's being developed and supported by the data Science Institute and not only are we supporting foundational research but we are developing various training programs Schmid Futures is supporting a post-doctoral scholar or postdoctoral Fellowship that's supporting 20 different postdocs across the Natural Sciences who are all focused on learning how to best use AI to accelerate scientific discovery the Tom and Margo pritsker Foundation sponsored a conference on AI and science that was held abou

t a year ago on the U Chicago um camp campus that brought experts from Microsoft research and Ai and science as well as people like Eric Schmidt who came and and gave a great uh panel discussion with us overall the training activities that we're developing are catalyzing groundbreaking research and accelerating Workforce Development so through these initiatives because of the support of the data Science Institute we have supported 67 different faculty-led research projects that bring together ne

w groups of people from across disciplines we've supported 208 trainees from 66 different institutions at our summer schools and the conferences and workshops that we've produced have yielded thousands of views from around the world of people really interested in understanding this research it's also yielded really big Returns on the investment when it comes to new federal grants we recently were awarded at the University of Cho Chicago a National Institute for Theory and Mathematics and biology

and you might have read about the Chan Zucker biohub in both of these Endeavors understanding Ai and how it can be used in The Sciences has played a critical role and these things would not have been possible without the support and the initiative of the data Science Institute and they're also helping to put Chicago on the map as a Nexus for ongoing research at the intersection of AI and and biology so with that I want to just leave everybody with two key takeaway messages um the first one is I

do not have brain damage the second one is that under yeah thank you I appreciate that the second one is that developing applied AI without understanding the underlying data science foundations is like developing biotech with Biology um we just won't get as far and we won't move as quickly thank you very [Applause] much thank you Becca for that incredible personal story and as well as the work that You' have done to understand machine learning in ai's impact on science and Society our next spea

ker is uh Nick fster Nick is a is the director of research at the data Science Institute and the newow professor of computer science at the University of Chicago Nick's work in Internet Security and performance often has implications for policy and tonight he'll take us through some of his research initiatives latest projects and Partnerships thank thanks we're in class now uh I'd like you to uh if if you like participate in this Hands-On activity there's a QR code there you can actually Point y

our phone at that it will take you I don't know if people recognize this little thing thing anybody run this thing speed test okay if you uh if you if you please uh you know fire it up tell me tell me what kind of numbers you get if you just run a speed test for me um just as you get a number just just Shout It Out 276 276 111 111 okay 343 343 okay I think we're getting the idea here these numbers I I actually got 1,800 so I was like I've never seen that number so 10 back here okay 10 perfect ok

ay so this is basically this hopefully this illustrates a a point um I'm going to talk to you about a a research initiative that was enabled by the data science intitute called the internet Equity initiative and a big question that I think uh many of us have seen read about maybe even experienced right is poor internet quality right and a and a uh big thing that's that's going on in this in this country right now uh is um Federal efforts to improve that especially in regions in geographies and c

ommunities where people don't have good internet access where it doesn't perform well or maybe they don't even have it at all um okay well in order to basically uh to answer that question uh we need data about what your lived experience on the Internet is like and as you can just see with that short little exercise we just did um that was a speed test right I think many of you are familiar with it it's like supposed to tell you how fast your your internet service is and uh as we can see we get n

umbers all over the place right so we're dealing with the question how fast is my internet as we sit here in this room and the state-of-the-art tool basically gives us numbers that they're all over the place it's noise we we have real trouble making sense of that right now take that to a broader question like what does internet look like uh what does internet access or internet performance look like on the south side of Chicago or on the west side of Chicago how does that basically compare to th

e North side or how does that compare to New York City or downstate Illinois now we've got questions of uh test design measurement dirty data uh spatial sampling right how do I sample a city how do I sample a neighborhood how do I sample a state um temporal right if I ask you to do do it again if you want um you I'm sure you'll get a different number right so when I ask a question about what regions in this country are are underserved or unserved as far as Internet access is concerned we've got

a big data problem at our hands and that's the that's the problem the internet Equity initiative is trying to solve now this is not a problem I dreamed up actually this is this is uh right now there's something like $60 billion being invested by our federal government um to solve this problem now how do we how are we supposed to figure out how to spend that money well as we know the federal government has the solution to everything right they just throw money at it and and then the problem is so

lved right we've seen this story before but seriously that money has already been given out to States who now have to figure out how they're going to spend it where they're going to spend it on what are they going to spend it fiber on the ground Towers uh improving uh in infrastructure in buildings in multiunit dwellings nobody knows we need better data to basically help us answer those questions okay so we did a little speed test let me ask you another thing I uh about about speed testing so th

is is some data from speedtest.net um in three different cities Chicago Philadelphia and Washington DC does anybody have an idea of like what this shading uh shows this is this is uh pertains to speed test anybody any guesses I know it's like class nobody wants to speak speeds speeds great great idea excellent if I don't know if the this is speeds you know why because this is actually number of tests taken in those parts of of these cities and so what we see right is that on the north side of Ch

icago a lot of people running speed tests in the neighborhoods where we'd like to get more data about do we need to do we need to make investments is this neighborhood unserved is it underserved we don't even have the data right so this becomes a much larger question not only of of uh what does the data mean right what is that data telling us right as we just uh puzzled out but also how do we even get the data so we've got a gap in the data and we that actually presents us some really interestin

g sampling problems as well right because one of the things that we're studying with in the research initiative is is like well okay there's huge gaps in the south and west side of Chicago we should probably go get some data from those neighborhoods to help us understand like what uh what the State of Affairs is in those neighborhoods but we can't go knock on doors like to every household so how do we basically conr construct that spatial sample okay well I we kind of like how know how to do pop

ulation sampling I mean we've seen that go wrong in the past before with with polling uh but we sort of understand population samples infrastructure samples is a totally new can of worms how do I basically sample what internet access looks like in Southshore well I don't know there's two or three different isps down there there's different modalities of connectivity um we've got interesting discontinuity so just because this house looks a certain way and we don't even know what it looks like by

the way right CU we just ran speed what does that mean when we move down the block two houses down so figuring out basically not only how we're going to go get that data which is a systems problem a data problem but also there's a there's spatial statistics uh problems involved in that as I mentioned uh these are not questions that uh we dream up uh these are really coming from stakeholders um This research initiative started about 3 years ago um some of you might remember 3 years ago we weren't

sitting in a room like this at all but we were sitting at home on zoom and one of the questions this the city uh uh asked us was what is the Baseline internet performance level and speed that should be considered acceptable for remote learning well already you know now we've got a sense that like measuring speed is is kind of hard but now we've got another question right which is this is more about lived experience right this is like does my zoom call work does my zoom session work so now we ha

ve another interesting question that's like well how fast does my internet need to be to support those applications that I really care about because whether the number is 340 or 1,800 or 100 who cares right I just want stuff to work and so now I've got a different data question which is how do I basically construct the measurements that give me that data about lived experience experience and maybe are there correlations between the speed tests that we just ran and the things that I actually care

about which is like is this thing working for for the things that I want to do and so this question came from the city um and I I just want to underscore that not only did we help them answer that that correlation question but also this does result in in scholarship this this research actually became the the Cornerstone of a students PhD uh dissertation work uh the initiative itself has number of other activities um we've we've Run summer programs through the University of Chicago's office of s

pecial programs and college readiness program one of the students who was in that uh class Chris Deng um is uh got inspired I think by some of the work that we were doing and decided he was going to do uh an advanced placement research project uh at Walter pton uh where he's where he was a student in CPS and he did a study uh using some of the the the data that we've gathered across the city and some of the data that he also gathered through doing his own deployments to understand how internet a

ccess varied in CPS schools in different parts of the city north south and west sides Chris I should mention not only did this project but as a result um got uh really into computer science and now he's um a first year at the University of Chicago He matriculated this past fall uh first generation uh college student yep um the uh the the the initiative has also had a number of other Outreach activities including open source software packages um as well as uh engagement directly with the city inc

luding on the digital Equity Coalition there's a couple of uh transitions to practice that I'll briefly mention one is there's the the $43 billion uh part of the 60 billion invest being invested as part of the Broadband Equity access and deployment uh um uh process from the federal government communities can challenge what the Federal Communications Commission says about what internet access looks like in those regions and uh part of that uh process requires people to basically gather data about

their internet connectivity and uh submit uh data to say no actually what the FCC thinks about internet connectivity in our region is not actually the reality as part of the initiative here we've built a site called B challenge.org you can go home and help your community by taking speed tests uh at be challenge.org and if they're not up to Snuff submitting those to uh to uh our local government to uh to perhaps help get your community better internet access um there's also another transition to

practice which is a lot of the technology that's been developed in this uh initiative has now become part of a commercial um uh uh Venture called net microscope that was uh I don't know if I think uh psky is here but uh that was actually something that went through uh the psky center through the iore program through the compass accelerator and then uh ultimately through the George Schultz Innovation fund uh and I should say that we're hiring uh okay so thank you very [Applause] much thank you N

ick for taking us back to school for telling us how to clean messy data but I have to say some of us are really upset because the speed I'm getting in this room is 1if of what I get High Park so now I need to go and call T-Mobile some of you have the same experience there is a lot of trauma in the room now but let's move on so I'd like to introduce the next speaker David uminski David joined the university in September 2020 as the executive director of the data Science Institute before joining u

s he was an associate professor of mathematics and executive director of the data Science Institute at the University of San franisco where he was also the founding director of their undergraduate programs in data science now now as the executive director of the DSi David oversees all activities programs and Partnerships both within and Beyond the university [Applause] David good evening everyone all right make sure this works um it's really inspiring for me to hear from both my faculty and inte

ract before the presentation begin and I'm really looking forward to talking with all of you after around this topic I have to say I've devoted exclusively most of my career that wasn't spent proving theorems and Mathematics on this particular problem of data science um today I'm going to talk about the parts of our Institute that are focused on our impact and Outreach in the community but what I want to say is what you heard already is that work our research done at the DSi and at the Universit

y of Chicago drives everything we do we have world class faculty you've heard from two you're going to see a preview of many more there's more in your booklet but every one of these incredible people come and bring what they know to work every day so that we can drive the impact in Chicago what we just heard is a research integration of our core research with our community imagine faculty that sit with the mayor's office and talk about hard problems now think about what Becca was doing in Ai and

science bringing the University's bearing and Leadership and driving impact and investment into Chicago itself the Chan Zuckerberg uh biohub the NMB this is just the beginning of what you Chicago's leadership it means to the impact of Chicago itself in greater Chicago land Dan just said and and outed me that I'm I'm new to Chicago I moved here during the pandemic I was convinced over a zoom call that I should just pick up and leave sunny California and move here and and and hope this is a good

decision for me and my two girls right and my wife so it turned out it was one of the easiest decisions I could have made and part of the reason why it was so easy is the work that we we build around what we do I want to tell you just about one partnership one of these Partnerships I'm the most proud of and we've been working for so long with and that's with the city colleges of Chicago I'll say CCC from here so what you see up here is an opportunity that could have only been led by the Universi

ty of Chicago but only could have had impact in Partnership when I had to take a call 2021 first year on the job it was a call over Zoom a lot of you Chicago folks a lot of folks from the city colleges and really quickly we learned about an opportunity but also a risk and that risk was there's 7,000 undergraduates of the University Chicago that are going to get the state of-the-art access to education research in Ai and data science but there's 70 to 100,000 more students in the city colleges th

at did not have that access and for us it was an incredible opportunity to understand do our ideas translate does our core concept our mission our vision have impact where it matters in our community so right away faculty from both institutions began collaborating on understanding what was possible at the city colleges very early on we hit one barrier that barrier was and it depending on your Workforce it's really hard to recruit topf flight teaching faculty in the area of AI and data science ex

perts in general in this field are hard to come by it's a new field so what did we decide to do we didn't throw our hands up and give up we decided let's roll up our sleeves work together and let's build that Workforce together so we began a program called the preceptorship program an idea of bringing fresh new phds graduates from all across the nation to the University Chicago and and CCC to develop that field of researchers Scholars and teachers that know the very best techniques in teaching i

n the city colleges as well as the University of Chicago that was the idea uh you hear you see up here a quote from the chancer uh of of CCC that I think summarizes best often when you see a partnership between a place like the University of Chicago with a stored history a prestige a place in the world partnering with a place like the city of colleges it is not mutually beneficial it is not always bidirectional and it is not always without extractor practice when we put those things on the table

we build new things that build capacity for both our institutions one of the things I didn't tell it was also hard for the University of Chicago to recruit Top Flight teaching faculty so here we are now 18 months into this partnership we've got five preceptors I'm going to show you their beautiful faces next slide we have over 600 students in growing that have been taught in this program both at the city colleges and the University of Chicago across four of the seven campuses we've also now beg

an to attract the first research funding from the NSF because the NSF was wondering how did you even develop this model of interaction ction how did this partnership even beg and that's what we're studying there we have a lot to do and right there I have a lot of credit to give right now to Walter uh to to our good friend Massie who has been a a steward of both the University of Chicago as well as the city college that really was our support when the going got tough these aren't as easy to to to

stand up when we um when we got going so as a result of this though that was one partnership that part parip resulted in a conversation that expanded way outside of that is what else can we do as a city and that was very easy to come up with every University that we began having conversation with said I also want to talk about what is the Chicago strategy for data science and Ai and right now we've now convened at the University of Chicago 50 plus faculty across these universities thinking abou

t things like transfer Pathways thinking about the curriculum that we're developing at this University and and how curriculum like it serve their students those faculty know how to serve their students best we know how to serve ours best and it is the conversation and the partnership across the entire city that gets me really excited about impact now we're talking about lots of students and we're talking about a vision that has sustainability and capacity building from the beginning this is exci

ting for me uh you know one of the things I really want to talk about in all this process is really our partnership with industry and Workforce many of you have come back to this room and have come back to the University from a place high up in Industry I am very grateful for the Partnerships we've had already Morning Star has been incredible partner in venergy many other partners you'll hear about in just a few moments from my faculty leader uh Mike Franklin around this they provide opportuniti

es for our students to grow to thrive and to contribute to what the core Frontier of data science and AI looks like in industry and it's impacting all of these instit institutions for me that's really fun and it's a reason to get up every day but what I thought I would end with here is actually well I guess I have one more slide I said I end regionally I like what we're doing but very early on it was clear University of Chicago occupies a Global Leadership position and right away very early on w

e thought about what is our national network of impact can be as we build our research as we build our education programs and we just decided intentionally to build a new network with places with their own stored history that looks similar to ours in some ways and different in other ways places like Morehouse College places like Howard places like UTSA and Fresno State two of the five largest Hispanic serving institutions in the entire country we convene faculty across the grant supported by Roc

kefeller foundation and MasterCard to think hard about how do we think about social impact data science how do we do research in it how do we think about educ ation programs in it and after these eight institutions convened for the last two years many new degrees many new programs and hundreds upon hundreds of students have now been engaged my favorite here of course and I'm very proud of this because I was around this summer was the culminating experience this summer was bringing students from

all of these institutions along with our U Chicago undergrads to work on Research hard research problems in applied and social impact data science in the environment in human rights uh in Ocean health and for me when they were finished that work they said in their exit survey that was one of the hardest things I've ever had to do and I was like well welcome to the University of Chicago the other thing I would say about it was as soon as they were done Blood Sweat and Tears they went out and talk

ed about their work they talked about the research at conferences and half of those projects have already run best awards at those conferences based on that work so I'm very proud about what's going on and it's just at the beginning I'm going to end though with this is what I meant to end with is not a story about big programs and big Ambitions but about one person because I think it's really hard to talk and and understand what it means to do data science at a national scale or a regional scale

or even a city scale so let's just think about one person one of the most important areas that we started very early on thinking about is the actual way to get into this field we heard from the city colleges saying they're worried about their students being overlooked in this opportunity and we roast the occasion before you get to the community colleges and before you get to the University of Chicago or any of those other institutions of higher ed you are first a high school student which we al

l were however much we liked it and so if we don't go into our community into our high schools and show them what the potential is what the future is of this field and encourage them that they belong that they should be there and they should engage then we've missed another opportunity for the last 2 and A2 years we've had a program called data for all it has served by and large CPS students from the south and west side but all across Chicago as well students now with hundreds of applications pe

r semester we let in 20 or 30 they come they spend 3 to four hours on a Saturday here at the University of Chicago and we are teaching them data science science and reasoning how to think about code how to think about algorithms how to make impact it's Project based so the students know when they're done they know how to make impact and one of those students that was in the very first of those cohorts is a student by the name of Jalil he went to Kenwood academy uh and that's a block and a half f

rom my house he went to the first class of data for all which unfortunately was on Zoom because that was also what we did during that time and Jalia you know he sometimes you know he wasn't sure he wanted to be there and he wasn't sure that this was the program for him but through a lot of intention and good practice we hooked him he got really excited he applied directly to our summer lab program which includes high school students this is an applied research program where you come in work with

faculty and solve a hard research problem in data science and Ai and we let in high school students in this program so if you have a high schooler I welcome you to apply for the summer After Next the applications are closed now uh and jalo applied he got in he contributed he solved one of the hardest problems that we had that year in computer vision he was the first order contributor to that problem and the story I would have liked to tell you is he came to the University of Chicago right after

wards that's the part I'm missing I think there was some fa I think there was some familiar pressure from parents about going to Michigan instead but it wasn't because he didn't get into the University of Chicago I'm very proud of stories like this I think I want you to know know there are hundreds of students like this right now that the University of Chicago is thinking about making impact on and we're going to need your help to do that all right thank you very [Applause] much thank you David

for that incredible story on our Partnerships to provide closing remarks and moderate our Q&A session I'd like to invite my fellow faculty codirector uh Mike Franklin Mike is a leading researcher in large scale data management system he is the Morton deall distinguished service Professor for the Department of computer science and Senior adviser to the provos for computing data science [Applause] Mike okay oh you forgot this part then you wouldn't have had to list all the titles okay great so um

I hope uh you know you've gotten the idea that we're we're at a really um exciting and unique time where uh technology is just plowing ahead uh things that seemed like you know just crazy dreams just a few years ago now seem very possible and that's having tremendous impact as you as you've heard from you know the various faculty you know really across everything we do at the University whether it's research education Outreach and so on but really all across Society so we've talked about the his

tory uh and a little bit of the context of the data Science Institute and um before opening up the Florida questions which we'll do in just a couple minutes so think of your questions um I just want to take a couple minutes to um look forward a little bit tell you uh where we see things going so I think the best way to give you a feel for the momentum we have and the Ambitions that we have is to talk about three different things uh faculty faculty hiring uh which we've been incredibly successful

at uh our plans for for research going forward and um uh a little bit more about Partnerships so let's talk about faculty so um we've been on a hiring tier uh as has been mentioned several times it's really really hard to find uh people uh with expertise in Ai and data science uh at at all in the world and to find ones that are willing to take the pay cut that one would have to take uh even at a well-funded uh institution uh to be an academic rather than going to Industry makes that pool even s

maller and I'm really happy to say that we've hired really one of the best young groups of um Ai and data science faculty anywhere in the country um and so these people uh that we've hired over the past few years uh come from really leading places uh in the country whether it's you know MIT or Berkeley or dare I say Stanford or Carnegie melon uh they've worked and some of them still are working at companies like Google uh meta uh Amazon and so on and uh some of them are active and and successful

entrepreneurs as well and they're bringing uh their experience and their entrepreneurial expertise to the Chicago ecosystem so um some of the areas that these people are working in and if you're in any of these I'd be happy to tell you afterwards you know who to talk to uh but human centered and trustworthy AI we've heard a little bit about that uh during the evening large language models things like chat GPT why they work how they work how to improve them when you can't trust them and so on uh

scalable data management systems my favorite topic uh distributed and Federated learning for doing um AI kind of across large networks similar to some of the things Nick was talking about uh economics of data right data is an incredible uh resource right it's the new oil um so what are the rules of the of that economy how how do you build uh viable data markets and uh of course as Becca mentioned uh what are the theoretical underpinnings of data science and Ai and how do you use that to to buil

d all these things on top of so um they've done a lot uh you've heard some of the accomplishments already uh I'll just mention a couple that you uh haven't heard about um recently uh a group at you Chicago released uh a couple of tools um one is called glaze the other is called Nightshade um and what these tools do is they excuse me they help artists protect their work from being uh appropriated or misappropriated by generative AI uh without their without their permission so um uh these to tools

have been released as open source software they've been downloaded in the last few months over 2.2 million times um they've received uh coverage in you know all the major media outlets and uh have really started a debate uh in the U Chicago Fashion of what are um acceptable defenses for people like artists uh in this world where there are companies that are trying to use their their creative output and not necessarily giving them credit or compensation for it and so um this is uh a project that

uh uh I think you'll hear more and more about in the future uh we uh I think the other one I'll I'll mention is uh some work that one of our faculty aloney Cohen who's up in the top left there has done where um he's um basically shown fundamental flaws in algorithms that many many compan companies and people are using toidentify data right so to make data private uh in order to uh meet the the the requirements of data privacy mandates in the U uh in the in in the uh European union and really ac

ross the world and he's shown uh fundamental reasons why these approaches that people are using don't work and will never work and again this is a very U Chicago type of a project where we're getting at the theoretical underpinnings of a really important practical problem so um that's a little bit about our faculty and again if you want to hear more about any of those topics we're happy to discuss those um in terms of research you've heard about our ongoing research initiatives in uh Ai and Scie

nce in Internet Equity Dan mentioned briefly our work on data and democracy with uh the Harris School of public policy and the political science department um which is looking as things as you'd expect of Missin disinformation as well as uh trying to understand how to be a how to use data to enable governments to better serve their constituents um but maybe more exciting and more future-looking is we we recently put out a call for proposals across the campus for people that want to start new int

erdisciplinary research uh initiatives we got um nine proposals that you can see there and areas like uh AI for climate AI for medicine um you know um um uh human centered Ai and so on and um we're in the process now of reviewing those uh based on the amount of funding we have we will we'll will'll award some number of these to try to get these uh these projects going and get these interdisciplinary teams people from very different parts of Campus uh faculty students researchers all working toge

ther on these important societal problems uh finally I just want to mention um in terms terms of partnership uh one of the Partnerships we haven't said too much about so far is our industrial engagement um we strongly believe that in order for our work and the work of our students to have real impact in the world and in the Chicago area um we need to be engaging very closely with leading companies and so um we have a number of initiatives as you imagin just projects you know that are indiv indiv

idual faculty doing sponsored research or or joint projects with companies but we have two other uh things I just wanted to bring to your attention one is our industrial affiliate program um which is built to connect people from companies mostly in the midwest but uh more broadly than that uh with the excellent work that you've been hearing about that's going on in the data science uh to make those connections uh to um to uh uh build networks to facilitate the exchange of ideas between the unive

rsity and what's going on uh in the commercial world and you can see the the companies that are currently involved in that um so this is something if you're involved in a in a local company and you're interested in please talk with us um and then I'll just mention briefly we've created something called the transform incubator oh sorry accelerator uh jointly uh uh with the psky center for entrepreneurship on campus and what transform is is it's a a 16we program uh companies from all over the worl

d can apply um they get to work with U Chicago experts and other business experts from around the Chicago area and they get training on how to take their ideas or their initial prototypes and go down the process of turning that uh into a company now transform itself is a startup we just graduated our second cohort um and one of those companies a company called um uh yeah Echo Labs um recently it was just in the news closed what um what is being called the largest preed round ever in Chicago Hist

ory it was some $8 million or something like that so and uh we taught we taught them everything they know no not really um we're currently uh in the process of reviewing applications for cohort 3 and uh I think you'll be seeing more and more interesting and exciting companies uh coming out of this program so with that um I hope we gave you an idea of uh where we see things going and and how we got to where we are today and what I'd like to do now I know uh you all have a lot of questions about d

ata and AI so I'd like to invite our uh my faculty colleagues up to the stage and uh open the floor to questions so uh faculty [Applause] colleagues okay so who would like to ask a question of our our experts uh in the oh sorry in the back I see somebody hi hi uh my name is Alexis and I am the director of the internet Equity initiative um thank you all for your uh very quick lightning talks um having been at University of Texas and then at University of Michigan before this um I had the bias tha

t private school would not have applied research like a public school would and since coming here I've been completely surprised with the depth and breadth of applications and not only that but inclusion of students and so I'm curious now that we're in the Q&A if you guys could share your favorite example of how you see applied research and the University of Chicago data Science Institute integrating with each other one of the things that I've observed this is this giant Gap uh between what is e

xpected of modern corporations governments and nonprofits and a lack of expertise and so borrowing expertise and training people up on that expertise I think that's where we're going so I would love to hear um what you guys have to say about that thank you I'm going to let you guys self organize go I'm going to turn on I'm going to turn on my mic um I don't mind going first I think um I have the pleasure uh or or the pain of so I've really enjoyed setting up this Clinic project this experiential

learning piece where students go and then they take all three or four years and if you're an undergrad you know of deep Theory rigorous study intense systems education and go try to solve a problem so we now have integrated in all our education this opportunity um in terms of really world real world applied impact I'm fortunate enough to be the faculty uh mentor of the project we do with Morning Star right now in this process and there we've had a team of students working the state-of-the-art o

f what's known about generative AI Lang large language models and understanding how those skills and new breakthroughs actually have application in Industry I won't tell you the punch line yet but it turns out there's a lot to do there and what I'm excited about is having partners like Morning Star really interested in actually authoring research around this so right now we have a paper under review of the work that's jointly authored by our students our researchers and researchers at Morning St

ar now on that work and so I can't think of a better example of marrying those two things as closely as possible for for everyone's benefit anyone else need to jump in no I don't feel obligated I'm happy to I yeah experien the students at at E Chicago are deeply interested in applications and deeply interested in doing research that's going to to better Society um many of my students are are super interested in understanding how we can use AI to improve climate science because they're very conce

rned about their futures um not only their Futures but the futures of their families and communities um and so I think the kinds of things that we're doing in the data Science Institute and the impact on society is is extremely active the students great I I think one of the things that uh really had a a big impression on me about 10 years ago was a keynote that one of my colleagues gave and he at at at a conference in my field where he said there really does not need to be uh a tension between d

eep scholarship and affecting the practice and that really had an impact on me uh and I think uh what I tried to to basically impart heart to students and what I see is is really imbued in the research that that I see at you Chicago is is that the the work doesn't end with scholarly publication the work in in many cases that's the start right that's how we start affecting the practice and I think I I can't pick just one example because I think everything we heard tonight embodies that is is is t

he Deep scholarship and affecting the practice through through policy through Clinic through entrepreneurship um yeah I think I think we heard many many examples of that great uh any NE next question doesn't have to be about what we were pitching I mean really anything you're interested about whatever yeah uh I noticed that uh you didn't list the law school in the uh list of you know cross team collaborations I would have thought that like the large language models and uh the UFC law school I th

ought would be a natural uh collaborative fit and I'm wondering whether that was uh whether there is any any interaction there going on I I can speak to that uh in the data and democracy initiative that that Mike talked about there's uh there's uh some collaboration with the law school particularly uh in the area of uh in the area of content moderation so uh Mike spoke for a minute about things like misinformation disinformation which um I guess we'll probably hear lot more about this this elect

ion year uh but uh one of the one of the topics that comes up there as I as as I know we've all sort of experienced is um how do platforms that host content make decisions about what content gets to stay what doesn't get to stay uh or what are the other dispositions for that content and the users who decide to post them and as it turns out it's kind of the Wild West uh but it also varies uh and there's a there's a there's an intersection with the law there right because if we talk about things l

ike copyright right there's a lot of there's a lot of uh legal Doctrine on on copyright in this in this country the dmca etc etc now you go to something like disinformation um less so right and there's some very interesting work that's going on in the data democracy initiative to look at basically how um different content platforms articulate uh organize and also enforce those those uh policies and how how those differences sort of intersect with with uh legal precedent or or lack thereof in man

y cases so that's that's one example but I think um many of the faculty uh that that Mike listed on his slide uh Looney Cohen I know uh teaches jointly with the law uh professor in law law school leor stovitz a course on privacy um chenan I think the work I mentioned from Maloney was actually done with from from the law yeah so that that work that I so there there was that work the work that I was just talking about was was uh collaboration with Genevie lakeer who's a First Amendment scholar so

leor stovitz Genevie Lake here um Jonathan mour also has been involved in uh some of the work on uh copyright and generative AI um we've filed some uh comments to the USPTO on that so there's a bunch of crossover to the law school right we also spent a lot of time talking to the office of legal counsel before The Nightshade program was released because the what what that does is if you train your model on it it poisons your model so it doesn't work on other content but but I do think this is a a

n area that's just really ripe with potential and I think the example that Mike mentioned earlier with aloney is is a perfect example right there were ex or there are existing privacy laws Hippa in the United States for healthcare law or gdpr in the European Union and he said hey what what does this actually mean in practice if we try to enforce these laws and it turned out that the common legal standards do not protect people's privacy and I think that just illustrates that some of the the risk

s that we face if we start thinking about uh designing regulations or certifications for AI systems without really having people with a deep technical knowledge intimately involved from the very beginning um and so because of that I think you know your question is is perfect like there's a lot of opportunity for interactions and collaborations there question one right here hello um I was wondering many AI programs use aggregate of past research for their programs which inevitably contain biases

towards race and gender how does University of Chicago's data Science Institute combat this in their AI programs so certainly in our educ ation programs um which which David and and Dan highlighted um this is covered we give many examples about you know not just what the methods are but the ways in which they might be misused and I think that um you know it's it's more complex than someone might initially think right I think some people might say oh as long as I have a sufficiently diverse data

set I'm fine problem solved um and that's not the case at all right there been a number of of problems identified with data sets like imag net um which has been widely used to train a lot of the modern computer vision tools that were sort of hidden under the hood right but once you start daiving deep you you see like major problems like you know a woman just sitting on the beach and one of the labels that we're using to train these image recognition systems is like klepto Maniac um you know just

totally inappropriate um but you know also I I'll just highlight the work of our colleague at the booth school um sindel Mulan who is looking at a um an AI powered system used in healthc care context where what they wanted to do was predict you know how much um Health Care needs people had and they had a really diverse pool of patients that they trained this on so problem solved right and then what what he showed was in fact that the resulting system was extremely biased and what happened was t

hat the people who were developing this said well I don't know how to measure healthc care needs so what I'm going to measure instead is Health Care expenditures but if you know something about health care in the United States you know that expenditures are not needs and the correlation between them varies a lot across different socioeconomic and ethnic and racial groups and so by making this sort of design decision about how they were going to set up their training they inad certainly put a lot

of bias into this system um and so I think you know thinking carefully about these case studies highlighting them in our classes um when we have reading groups within our research Labs going into the details about what caused these problems and making sure that we're we're not repeating the mistakes of the past is just integral to to everything that we're doing here in E Chicago and I should add that we do this from the first class students taking data science s in the first class we do I I sho

w them an example of machine learning algorithm that can predict Age based on molecular data and they get very excited because they have this tool and they say wow I can look and sort of try to predict my age or my biological Age based on that and I tell them the story on how this is used in Germany for example to predict the age of immigrants and then slowly I let the students destroy that example by finding the fault in using a model that's trained on Californians on to be used on immigrants f

rom Middle East in Germany and the impact that has on practice there are legal aspects policy aspects of it so that's from the first class they take great all right I think we have time for one more question anyone want to ask last question oh yeah all right great last one so exciting that what's going on and and moving to impact and certainly impact through I think through this this evening we I've heard about two startups so I need to come talk to you and learn about a few more but so I would

imagine that creates some tension because now we have very creative faculty starting companies maybe wanting to go work on moving that effort forward what what how are you managing that or what kinds of policies are you putting in place with University to help facilitate that you can take that panelist oh I get D you're involved let me take the easy one which is our students right I think the students are our future we sometimes we think we all want them to become faculty at MIT or eago but we k

now in fact looking in this room that there's a many measures of success so when it comes to the student ideas they own their IP they think about what they want to work on and we actually built things like transform and through the psky Innovation Center and our and and the culture itself to incentivize and support them you know when I I went to some schools where you didn't want to tell your faculty you wanted to go to Industry and you that was something you did after you graduated you know bec

ause they thought you were going to be a postto and and go on and be a researcher and I I'm very proud to say that there's a much more holistic approach to this field and what we're doing and I think it comes a little bit from the nature of the work we're doing the the line to impact is so short you can really have a great idea and in six months go get funded or less 16 weeks and so for us on the on the student side our graduate students even our postdocs I want to build a culture that incentivi

zes translation and impact broadly through entrepreneurship and partnership I think with our faculty it's a bit of a challenge because we we actually want Nick to launch his company and others and we have great examples of this type of of behavior happening that has great impact for the world and for our students and for our research um but I think you know it's it's a complex question of like if it's so successful will they come back and I think that's kind of what you're alluding to yeah there

's a lot of discussions at even at the highest levels of University about how to how do we be more open to to this because in this field that's where you know Innovation happens at that intersection between great new ideas and and putting them out in the world so one thing I didn't mentioned but one of the faculty that was on that list uh that we hired uh I talked about uh you know this big preed round in Chicago well he founded a company uh out on the west coast that just closed uh $105 million

series a um and now we've got to convince him to come back and teach a class so you know we're we're we're living this and uh the good news is that the university is looking at this as an opportunity and trying to figure out ways to make it work so great so with with that um I think I just want to thank everybody for coming I hope we shared our excitement and what we're doing and and uh the importance of partnership and your support um we have a reception in the room where we were before the uh

before the program started and so please join us and and uh all the speakers will be there and we look forward to the conversation thank you very much you