[MUSIC PLAYING] [MUSIC PLAYING] So welcome, everyone, to our second round of Training Hub Fika sessions. So today, I will talk a bit about learning outcomes and assessments and how we best do this, specifically in short format courses, which are most of the courses that we have. So learning outcomes, what is that? And why do we care? And what do we do with them? So basically, a learning outcome, we start from the end. So we start by specifying what the learners will know, do, or be able to demon
strate at the end of the course or the learning experience that we have. So an example could be, by the end of the course, the students should be able to create their own Python functions. Then we have been very clear what the students should be able to achieve by the end of the course. And typically, a course has several learning outcomes that the students should be able to achieve by the end of the course. And why does this matter? Why should we have learning outcomes? So to start with, if you
have well thought through learning outcomes, it will help you in your course design, because you already know what they should learn. So you start with saying, these are all the things the students should be able to do after the course. Then by default, you know what to teach them to be able to be able to reach those goals. And it also creates a scaffold for choosing what type of learning activities that we want in the course. And we will come back to this later. But depending on to what level
someone needs to know something defined by the learning outcome, you can choose the appropriate learning activity. Should it be a lecture? Should it be an exercise? Should it be a group exercise, for example? And thirdly, it makes it very clear to the students what to expect from the course. They can directly go to the learning outcomes and see this is what I will learn at this course. And especially if we deal in, for example, a short format course, a one week course that comes from SciLifeLab,
often our students are people working at the universities as a PhD student, a postdoc, or a staff scientist, and they want to learn a specific thing that they need in their future work. But then defining and looking at the learning outcomes, they can see exactly, yes, this is a good fit for me. This course will teach me exactly what I want to know. So how do we then create these good learning outcomes? We talk about action verbs and something called Bloom's taxonomy. So Bloom was a teacher, res
earcher, I think maybe in the '60s or something, that came up with this pyramid. And these are verbs that relates to the cognitive processes in the brain. So looking at the lower level of Bloom's, we have the remember and understand action verbs. Or remember can also be repeat, name, list. There's a few different verbs that all kind of describe the lowest level of this cognitive process pyramid. And to put that in context, for example, if we would talk about carpentry and we would design a learn
ing outcome that targets the lowest level of this Bloom's taxonomy, we can have name some tools used for carpentry. So something remember, remember a hammer, remember a saw. If we move one level up to the understand, we would have or describe, explain, identify. For what do you use a hammer or describe how a screwdriver works? It's one level up, so you need to understand the tool in a different way. And then moving up further, we have the apply. And here we are on the actual skill set. So can yo
u divide a piece of wood and join the two pieces using a nail? Moving up further, we have the analyze. For example, what is the difference between a Japanese saw and a cut saw? And how do they compare? Or we can start to analyze the knowledge we have and do something with it. Evaluate which saw would you recommend. Then you need to know the different criterias or the different things related to the types of saws. And in the top level, we have the create action verb, which could be what alteratio
ns would you make this screwdriver better for something. Then you really need to understand the entire pyramid to be able to create something new from it. So this is how we can, on different levels, specify what we want students to know after the course is done. Do we want them to just be able to remember something or do we actually want them to be able to create new things with the knowledge they have? And as I already said a bit in the beginning, that different learning activities support diff
erent levels of Bloom's. So looking at this, for example, we see that lectures and webinars, we have them on the level of remember and understand. So they actually target quite low level of Bloom's. And the lectures and the webinars is what is most common still today in the university. But it's really hard to reach the high levels of Bloom's, like the cognitive process, to really learn something with just a lecture and a webinar. Going up a bit, we have the exercises. If you have exercises conne
cted to your courses, flipped classroom, for example, then we're somewhere there around apply, analyze. But they need to apply their knowledge or analyze it a bit to be able to do the exercises. Moving up to the highest Bloom's level, then we have the peer instruction, group work, or problem solving, or problem-based learning, where they actively have to use their knowledge to be able to create and evaluate and attack problems from different ways. And this is what I said. If we have written a le
arning outcome that wants the students to be able to create something, then we need to make sure that we have a learning activity that actually teaches them to do that. And we cannot just have a lecture and expect them to reach the highest level of Bloom's. If we look at an example here, we go back to our programming course here. So we have set one of the learning outcomes to say that by the end of the course, the students should be able to create their own Python functions. And here we see the
action verb is create. It is in the top of the Bloom's pyramid. That is our goal. That is where we're going to end up. So how can we now design a learning activity to make the students reach this? So first, we need to start at the bottom. We need to go through all the levels of the pyramid. So we start with the remember and understand. And we said before that a lecture is a good way to reach those like the remember and understand levels. So we can have a short lecture about functions. We can als
o-- we can ask a question during the lecture, for example. So here, there's an example of a function. And then I can ask, what would the value of the variable 'my value' be in the function? And then I can use that to create understanding of how the input into a function works to be able to go up to the understanding level. And once we have done that, we can have them do an exercise. Now we're in the analyze apply field. We can, for example, have them give them the task, make the function return
the value 8. So this is still only using the existing functions. So they apply the knowledge they have to an existing function. But they still need to look at the function, understand what it does, analyze what would happen in different ways. And then finally, we can say, for this exercise, you will do this in pairs. And this is the problem you have. You should create a function that takes two user defined values as inputs and return the product of them. So we don't give them any scaffold or any
code. We just say, this is the problem. You are two people or more. Like discuss it in a group. See how you can solve this. And then you write the function. And then you're up on the create level. And by doing this step by step, we have given them kind of like the scaffolding in itself to be able to do more and more in the steps towards creating this function. So this is a super basic example. But it all comes from the learning outcomes and the learning experiences that supports the different l
evels of Bloom's. So now then, how do we know that the students have actually learned or achieved their learning outcomes? So we're going to talk about assessment. Why do we need assessment? Well, we need to know if students have actually learned anything. It's good for the trainer. I can have a very good course. I think it's a very good course. The students thinks it's a very good course in their evaluations. But if they don't learn anything, is it still a good course? I would argue not. So we
need to know, has learning actually occurred? Assessment is also a good learning activity for the students, that they have to recall their knowledge again. So it creates a learning activity. And you can also assess the appropriate Bloom's level. In the example with the functions, if someone says, oh, I know a Python function, do they know how it looks? Can they create their own? To what level do they know it? We can assess the different levels of the Bloom's taxonomy here. We can also vaguely ma
p this to the levels of Bloom's. This is a bit more grey zone than the learning activities, I would argue. That's why I put the exams, quizzes, and questions in class in the parentheses, because I will show later that you can actually use them on different levels. But normally, you would use quizzes or questions in class for these, like remember, do you know what this is called? Do you understand this? So it's quite low-level Bloom's. Hand in assignments can be a bit more applied or analyzed. Wh
ile group reports and assignments, there you can have a bit more time so they can go up to the create and evaluate in the Bloom's level. Although this is a bit more floating than this. So how do we actually do assessment in our short-format courses? So if we do university teaching, this is something that is included in the curricula. We have an exam in the end. Some type of assessment or summative assessment is needed. But in short-format courses, that comes from SciLifeLab, for example, for PhD
level courses, post-doc courses, et cetera, we don't have often the requirement of having it. So it's up to us if we want to do a summative assessment or not. But we don't have the resources often for correcting hand-ins or larger assignments or even exams. So how do we still make sure that we assess our students in a time or resource efficient manner? So there's a few things one can do. My favorite is still multi-choice quizzes with feedback. And here, I put that up to the analyze level on the
Bloom's scale. And you remember that is in the middle somewhere. And that's hence the parenthesis around the multi-choice quizzes. Because you can actually target the higher cognitive levels using these multi-level quizzes you just have to be smart about it. And instead of asking, is this or that type of questions, you would have to probe deeper into making them analyze a bit more. And then once they have been able to analyze it, then they should be able to give you the answer of the multi-choi
ce questions. If you have an LMS, for example, a learning management system, they often have quizzes built in. That is a super good way in short format courses to add. You can even have daily quizzes, have five questions per day that covers the topics. And then you can have feedback with the wrong answers and explain why it's wrong. And that also creates-- it becomes a learning experience. And you as a trainer can get direct feedback. Is my students actually understanding what I'm saying, or do
I need to go back and talk about something more? Other things are also if you don't have an LMS or if you have an in-class, you can just have questions in class. Either you can just do the standard by asking people. You can have something like a Mentimeter where you probe them and ask them for questions. Also here, you can if you want, reach up to the analyze level of Bloom's. It might require a bit more thought into the questions. And also the last thing that one could do is if you have a group
work with peer presentations, for example, that reaches the highest level of Bloom's. You can check how well they create things without costing much extra resources for you as a teacher, because they would do it in class then. This is, of course, up to how much time you have, but this could be a good option without costing resources. So this is just a sneak preview of this field. And if you're interested in learning more about learning outcomes and assessment and how to use that more in your tr
aining, we have a lot of guides on our websites. And more guides are coming. We will fill this up as we go. We also have a course, Train the Trainer, that we offer in this that is, I don't know how many hours, but almost a week long course in total, where we talk more in depth about all things, the learning outcomes, the assessment, the design, the learning activities specialized for short format courses. We run one now. I think it is full now, but check back next year if you're interested. Othe
rwise, is there any questions from you that you have? (upbeat music) (upbeat music)
Comments