Should we grade critical thinking? And Can we?

PUBLISHED: Sep 12, 2017

In This Episode.

In this episode, Dave and Steve tackle the controversial topic of assessment in education, examining the research about the relationship between critical thinking education and critical thinking assessment.  Learn about the powerful roles assessment plays in education as a whole, the value of including critical thinking in assessment, the need for assessment to be more than just a grade, and the importance interdisciplinary considerations.  

Episode Archive

Podcast Transcript

Voiceover: Welcome to the Critical Thinking Initiative podcast, we bring you research driven solutions to critical thinking education. Why? Because as Bertrand Russell said, most people would sooner die than think. In fact, they do so. And now your hosts Steve Pearlman and Dave Carillo.

Steve Pearlman: All right. Welcome back to the Critical Thinking podcast. I am Steve Pearlman

Dave Carillo: And I’m Dave Carillo.

Steve Pearlman: And the focus of today’s episode is going to be on the assessment of critical thinking, the grading of critical thinking. We have a lot of discussions with faculty and get a lot of questions from educators about the importance or at least the relationship between developing critical thinking skills and assessing critical thinking skills. And is there an important relationship there or is it something we can we can ignore?

Dave Carillo: Yeah. And this is one of the most challenging things that we do work with faculty about because it encompasses so much of the classroom experience, from process to pedagogy to defining critical thinking to feedback. It’s all wrapped up into one. And when it comes down to it, students are really, really interested in kind of grade. They’re getting the kind of grade they’re earning. And so it makes it that much more of a pressing issue for the faculty that we work with. So we’re happy to be here today to hopefully help some folks out and, you know, put our listeners on what we think is a good track for doing this kind of thing

Steve Pearlman: Because certainly anyone in education feels the pressure surrounding grading. However, we choose to handle that. It’s a factor.

Dave Carillo: Yeah, absolutely.

Steve Pearlman: And so also coming this week, we’re going to have a question from one of our listeners, Dave.

Dave Carillo: Absolutely. We finally, you know, we released the first podcast, got some great responses. We’ve gotten some great emails and some great questions. So we want to start that dialogue.

Steve Pearlman: I was actually really surprised at the number of downloads and questions that we got as a result of the first podcast.

Dave Carillo: It was a pleasant surprise. You never know what kind of response you’re going to get, but so far, so good.

Steve Pearlman: So far, so good. All right. So let’s jump right in to this question of assessment.

Dave Carillo: All right, Steve, where where do you want to start?

Steve Pearlman: Let’s start with what we think are five essential points that we want to hit over the course of the podcast today. And these are again addressing the sort of the relationship between developing critical thinking skills in the classroom. And how do we assess that? Should we assess that? Do we need to assess that? Yes. So here are going to be our points first. That assessment is the biggest denominator, in fact, of what students do. Second, that an assessment that’s not focused on critical thinking can actually inhibit the amount of critical thinking makes sense. Third, that an assessment of critical thinking can actually enhance the amount of critical thinking.

Dave Carillo: It’s an equal amount of sense.

Steve Pearlman: Fourth, that by assessment, we really can’t just speak about something that happens after students are already doing their work. It has to be part of the process. Absolutely. And finally, that a good assessment of critical thinking has to have a naturalistic element to it, and we’ll talk about what that means later on.

Dave Carillo: Great. They sound like good points. All right. Let’s see if we can get to him.

Steve Pearlman: All right. So let’s start off here because it’s actually really interesting. And we talk a lot of times when we’re talking faculty, they are questions are focused more about how do I construct an assignment or how do I do an exercise in class that can foster critical thinking in my students? And one of the things that we always come back to asking them is, well, what kind of assessment are you using for critical thinking? And the reason that we’re asking that is not only because we want to know the kind of way they’re conceptualizing critical thinking so we can best pair exercise in assignments with it. But in fact, because most of the research points to the fact that the biggest thing that will determine what students go and do in terms of their learning process is not what you assign or what you talk about doing in class, but rather the kind of assessment you’re going to have and the extent to which students are familiar with that assessment. Yeah, and

Dave Carillo: That throws that can throw you off as a teacher when you spend so much time on process, when you spend so much time designing these assignments, encouraging kids to think critically and so on. And then after they do the assignment and after they listen to your lecture or they work through your exercise, they just want to know what grade they got. And it can be disheartening at points. But one of the things that we’ve had some success in doing is turning the assessment into an advantage for faculty, because if that assessment can mirror very closely and make very explicit what you want in terms of that critical thinking, then then the kids are more likely to accomplish that.

Steve Pearlman: Let’s reference this study here by Strathaven, Doshi and Jansen’s.

Dave Carillo: Let’s go to that.

Steve Pearlman: So two thousand five study and it talks about students perceptions of evaluation and assessment in higher. A review that’s the title, and they cite a number of other prior researchers who are basically showing that the driving force for what students go ahead and do in terms of their educational process is contingent upon their understanding of how they’re going to be assessed. Now that understanding might be misguided, or it might just be based on how they’ve been assessed before. Sure, but their conception of the assessment is going to drive what they do. So even if therefore we go forward and put out a very thoughtful assessment that calls for students to do a good deal of critical thinking in it, it’s a well constructed and conceived assignment in that respect, if their students aren’t clear on how the assessment is going to value critical thinking, then they may not approach the assignment in the way that the assignment is designed.

Dave Carillo: Sure. It seems like at that point, what they’re suggesting is is, is that the students will actually sort of go back to what they know. They’ve been assessed on how many paragraphs their essay has for so long that unless you are very clear about moving past that, if you just ask them to do it in the assignment, they’re not necessarily going to do that. They’ll just revert to that five paragraph. So that’s kind of what that research.

Steve Pearlman: That’s exactly what it’s saying. Yeah. And in fact, it makes me think of another study, and maybe I forget the the title of this one. So maybe we’ll throw it up on the website. But yeah,

Dave Carillo: That would be

Steve Pearlman: Great. This is a study where they took students. They had a control. But basically the experimental function of this study was they had two groups of students and they told one group they’re going to get multiple choice exam, and they told the other group of students that they were going to get an essay exam. And students went off and studied for these two different kinds of assessments. And then what they actually did was they switch the assessments on the students when they came for the test. So the students who said they were going to get an essay exam actually had a multiple choice exam. And the students who said we were going to get a multiple choice exam actually were given an essay exam. Well, lo and behold, the students who studied for the multiple choices exam tanked the essay. Atrocious essays. And the students who studied for the essay exam killed the multiple choice exam. And so the point here again, and I think this is actually pretty cool, is that assessment really has a really powerful ability to drive what students go ahead and do because the students who figured they had this more rich, complex, thoughtful assessment coming had to engage their study process more deeply. Whereas the students who felt that they had this more simplistic assessment coming that was only testing on rote memorization never gave as much thought to thing. So it’s really showing how assessment can drive what students do in terms of their process.

Dave Carillo: It’s really telling that if you think you’re going to be assessed on a single right answer, you’re going to look for that single right answer no matter what. And the students who thought they were going to have that multiple choice test didn’t bother with anything along the lines of relationships, complex relationships or nuance or various perspectives and so on and so forth, whereas in essay ideally might have all of those things and those students who studied for that essay exam just aced it because you can always sort of move downward on. Right, right. Yeah, they were able to comprehend more. They’re able to figure out more about the more simplistic multiple choice test. Yeah. Now that is telling

Steve Pearlman: And before our listeners, I guess, think that we are too hot on grades, we want to be clear about the fact that we’re not in favor of assessment for the sake of assessment. We’re in favor of assessment to the extent that it serves a warm educational purpose of actually driving students to do meaningful intellectual work.

Dave Carillo: And that’s a really good point. Yeah, we we’re in favor of assessment as far as what you’ll hear us talk about this and describe the kinds of assessments that we use as authentic. And what we mean by that is that if the assessment is less about just some arbitrary grade and more about bringing students to the next level in terms of their comprehension in terms of their drive toward completing assignments in terms of thoughtfulness, then you’ve really got something right. And now suddenly the grade means more not only to you and in your class, but to the student. And and it’s not something that you’re necessarily creating a hoop just for them to jump through, but you’re you’re allowing them to build a stronger foundation around that authentic assessment.

Steve Pearlman: And it’s kind of almost rolling us into the next point of ours. But I want to pull us back if assessments are so important, our most faculty doing assessments that contend with critical thinking. Do they have a standard for that study done by Nicholas and Rayder Roth, who looked at critical thinking assessments by faculty at two big universities? And they asked them basically. Do you use an instrument to assess critical thinking, OK, and generally speaking, the answer was no, not at all. And then for some reason, even though we don’t use this, we collectively as educators and obviously we’re generalizing here unfairly to certain extents. But we, as educators are then surprised or dismayed that our students aren’t doing the kinds of critical thinking that we’d want them to do. So we had responses to the effect of quote No, we have no formal rubric. It depends on the student end quote.

Dave Carillo: That’s a good one. I guess what they’re saying there is that if the student shows promise, maybe they are going to be assessed in a certain way. Whereas if you were going to assume that a student’s not going to be able to make the grade, maybe they are going to be assessed in a different way. Is that kind of what you’re talking?

Steve Pearlman: I guess. But then if and this is where it comes back to, right, if you’re a student in that experience, what is the metric by which you’re being assessed? How do you know how to develop or grow or what to present forward in terms of critical thinking to improve or to display your ability? And this is my favorite one, which was quote, We know it when we see it end quote just like the obscenity.

Dave Carillo: Just like, oops. Yeah, exactly.

Steve Pearlman: Yeah, right?

Dave Carillo: Critical thinking from that senator in the pornography hearings. I don’t know

Steve Pearlman: When it was a judge, wasn’t

Dave Carillo: It? I don’t know. I’m not

Steve Pearlman: Sure. I thought it was justice Blackman. Was it? Blackman could have

Dave Carillo: Been. We could look that up. We don’t have to put that up on our website, though. Ok, Wikipedia.

Steve Pearlman: I’m sure somebody will correct.

Dave Carillo: Absolutely. I want to go back to something that you said, though, in regards to the first quotation. And I’m hoping we can say a little bit more about this and maybe even revisit some research regarding this. But one of the things that I’ve noticed just early on, at least, that we’ve been approaching this from the sort of faculty to the student side. But that kind of confusion on the part of the student is is so significant can be significant. And it’s a huge, huge issue in terms of the other kinds of elements that go into strong critical thinking, classroom or ecosystem, as we’ve talked about. If the students are are confused about how they’re being assessed, they’re not necessarily going to grapple with process in the same way, they’re not necessarily going to engage the material in the class the same way. And that’s that’s that’s significant in terms of how we talk about assessment. You know, if the assessment isn’t clear, right, if the assessment isn’t asking for certain types of critical thinking, certain types of thinking at all, explicitly students can be really confused and it sort of derails all of the process, all of the assignment, all of the effort that that you’ve made as teacher in the class.

Steve Pearlman: And we can’t hold it against students. If grades are going to be on the table, then we certainly want to at least find a way to use that process to everyone’s degree, genuine benefit rather than something that’s just an impressive tool or something that’s just a rating or something that just goes in a book. Let’s make it something that’s pedagogically valuable and meaningful in the world. I guess we would ask our listeners the extent to which they actually use an instrument for assessing critical thinking, the extent to which students are familiar with it and so forth, rather than valuing critical thinking, but not necessarily really making it part of the fabric of the educational experience and assessment experience.

Dave Carillo: Yeah, sure. Or not foregrounding it as something that’s significant or important in the class. So a lot of times we’ll see assessments of writing assignments that, you know, just as many points to an essay that has an introduction as it does to an essay that has critical thinking in it. You know, twenty five for each write, that can be just as detrimental as having no assessment at all.

Steve Pearlman: Right. So many rubrics. I love the point that you’re raising here, right? So many rubrics that will make introduction 10 points use of resources 10 points analysis 10 points style 10 points grammar. So critical thinking becomes this thing that subsumed.

Dave Carillo: Yeah, with let’s back up again, because, you know, just like we want our listeners to know that, you know, we’re not advocating assessment for the sake of assessment. We’re also not advocating or suggesting that you can always free yourself up to the extent that you would like to to assess your students the way you would like to. So ask us questions about how to make these kinds of things work for you. We understand that you are not always just your own person in the classroom, and oftentimes educators have to answer to a host of assessments,

Steve Pearlman: And that takes us nicely, really right into the second element of this, as we’re talking about what these rubrics can look like and so on. And I want to reference this great study done by Bill Condon and Diane Kelley Riley over at Washington State, who have been. You know, in a lot of ways on some of the forefront of looking at critical thinking and

Dave Carillo: For a long time now, the studies from nineteen ninety four

Steve Pearlman: Two thousand four

Dave Carillo: Two thousand four for

Steve Pearlman: A little back story on Washington state and the study very, very briefly. And we’re going to centralize this a little bit, but they have a portfolio in Washington state and the portfolio has been being assessed on certain criteria. And then they also have a critical thinking guide that lays out some explanations for the students of what critical thinking is. And what they did is they went and looked at the extent to which critical thinking was growing and out of these writing portfolio outcomes and what they found was that the critical thinking was not growing at the writing portfolio outcomes. In fact, there was almost an inverse relationship between strong writing portfolio outcome and critical thinking. So the better writing portfolio outcome. We’re not doing as well in the critical thinking. And and why is that? You might ask Dave.

Dave Carillo: I might ask that I have a guess. I have a couple of guesses.

Steve Pearlman: Would you like to venture that? Guess?

Dave Carillo: Well, I think we’re going to find out that, you know, if we can assess an essay on how well it’s written and stylistically and how grammatically correct the sentences are. But that doesn’t necessarily mean the thinking is as strong and research tell us that, you know, when a student’s really grappling with an idea, you know, that can be messy, and the essay isn’t necessarily going to come out as pretty, it’s not going to turn out as pretty as as one might be if they were student. We’re just repeating what others say or spouting cliches or something like that. I don’t know. Maybe you might offer you my own. I want a prize. Excellent.

Steve Pearlman: So the essays were assessed on focus, organization, support, fluency and mechanics, and the essays that did really well in terms of those categories did not necessarily have much critical thinking in them at all. Because if that’s where you’re going, what you’re going to be assessed on, why would you bother with some of those other intellectual elements I see? And just as you talked about how critical thinking can make some of the writing messier, some of the more thoughtful essays scored low on the focus and the mechanics and the fluency and so on. Because, as you said, when students are grappling with an idea, the communication of those ideas, the eloquence can sometimes suffer, at least in the short term. And ultimately, of course, we want both. Yeah, right. But nevertheless, there is this inverse relationship between how the two were forming, and this really goes again to show how critical it is that if we want to have our students do well with critical thinking, we in some way create assessment that actually does that and that.

Dave Carillo: I mean, and that can be difficult at times. But you know, it just goes to show again if the essay or any assignment is just going to be assessed on whether it has an introduction, not necessarily what the introduction says and whether it has all the other formal components, you know, and whether the sentences are grammatically correct, then that’s I mean, that’s pretty easy to do without ever having to take a risk in terms of your thinking, right? And it’s pretty easy to do to integrate a quote or to if all you’re going to do is summarize that quote or just repeat what that quote that quotation says. So, you know, that’s one of the reasons that makes this study so well known is it is, you know, bringing to the fore like a very crucial sort of relationship between, you know, what students are going to do, you know, as opposed to what a lot of these kinds of writing assessments profess that they’re looking for.

Steve Pearlman: And in fact, there’s other research referencing Deutschland and Gordimer here that shows that for traditional assessments, traditional assessments would be things like multiple choice tests or papers that are graded where critical thinking is not explicitly explained and valued in certain ways. That quote students focus on getting through tasks and resist attempts to engage in risky cognitive activities. There is an ellipsis in there, so assessment can actually not just sort of derail but inhibit students willingness to engage in critical thinking if it’s not explicitly valued. And this is one of the things that we see a lot in our students here at USDA when they first come in is that we have to explain to them that their intellectual risk will reap reward, that we want to see that intellectual risk because they’re not used to taking intellectual risks. It’s much safer for them to summarize something than it is for them to really try to engage in an intellectual act about that work because there’s no risk of of being criticized. There’s no risk from them of saying something quote unquote wrong for lack of a better word. And they’re just in that mindset.

Dave Carillo: Oh, no case in point, not just with our students. Like all the students we work with on all the different schools, their minds are genuinely blown when we talk to them about how they can include in their essays, the things they don’t necessarily know. Out there sources or the things that they’re not necessarily sure about in terms of their own conclusions, right? You know, just being able to show them that they can articulate that kind of complexity or complication just floors them, right? Because otherwise a lot of students come to the table, especially in college, having been not necessarily, if not necessarily taught directly like it’s been suggested all along. It’s been implicit that, you know, pick a side and argue that side to death. And that’s not necessarily the case. So no,

Steve Pearlman: To embrace complexity to gray areas, the weaknesses. This is all part of what it is to be thoughtful.

Dave Carillo: So, yeah, I mean, and that’s and that’s one of the easiest ways that we can sort of make that link for faculty if you are articulate and value students ability to to talk about the limitations of their own ideas. Right. The gray areas, if you make that part of the assessment, they’ll start to think about the gray areas and the limitations of their own ideas. You stop getting kinds of conclusions based on one or two sources. World hunger is now been solved, and those kinds of things, or

Steve Pearlman: World hunger is bad.

Dave Carillo: Yeah, exactly. World hunger is, yeah, world hunger is bad. Food is good, and that goes back to assessment. Do they know what you’re asking them to do? Is it clear how they’re going to be graded?

Steve Pearlman: And on that same note, I want to talk about this study that I just saw, and I’m going to I’m not even going to try to say the names because they’re all Turkish. And actually, Turkey is doing some of the most interesting research or a tremendous amount of research now. They are critical thinking and teacher training, so shout out to Turkey. Absolutely. We might have to take a field trip.

Dave Carillo: I’ve always wanted to go to Istanbul. Is it Istanbul?

Steve Pearlman: It’s not Constantinople. So it’s called training of critical thinking skills and teacher candidates and placebo effect. A quasi experimental study, it appears in the European Journal of Educational Research in 2016. And so they broke the candidates down into three different groups. They had a control, they had a class that was the placebo group and they had an experimental OK. All right. Quote students in the placebo group were orally motivated for critical thinking in the general course of the lecture, not under constraints of a specific period. This motivation was provided through statements of encouragement, such as Critical thinking is important. Try changing your perspective and break your taboos and get the big picture. Ok. All right. So that was the placebo group. They weren’t actually given time to do any critical thinking. All right. They were just told how wonderful critical thinking is. And let’s face

Dave Carillo: It, we’ve all said those things.

Steve Pearlman: We’ve all said those things. Yes, probably should continue to say those things. Yes. The experiment group I’m quoting here was given critical thinking skills exercises created by the researchers before the beginning of the academic term. The application was repeated for eight weeks, sparing the last 10 to 15 minutes of the Class four activities. The aim during the activities was to have the students and suggest solutions for or express their ideas about a question situation directed to them that might have more than one answer. After the answer of the first student, the others are expected to negate it and to explain their own evidence. So what do you think happened, Dave?

Dave Carillo: Ok, so one class is at least being told the critical thing is important in the other classes are being told the critical thing is important, but they’ve got exercises, right? I don’t know. That’s tough. I mean, I mean, maybe I first guess might say that, well, you know, the kids that were doing the exercises might have been developing stronger critical thinking skills. Who knows?

Steve Pearlman: Well, that’s what we would like to hear.

Dave Carillo: We’d like to hear, right?

Steve Pearlman: But according to the California Critical Thinking Disposition inventory that they used, the group that did the best was the placebo group.

Dave Carillo: Interesting.

Steve Pearlman: So being told that critical thinking is important had the biggest effect. Not actually the exercise is now in truth. There were no statistically significant results at all from the study, but what was lacking from this was in fact an assessment of the critical thinking that the students are doing in the critical thinking exercises portion of the experiment.

Dave Carillo: Ok, so yeah, we could see so they could have been asked to do any number of critical thinking exercises and then just spouted off insanity

Steve Pearlman: As we all want to do it

Dave Carillo: As yes, we’ve all done that in the past. But since no one was really checking, no one was really assessing the strength of the critical thinking or responding via assessment to the strength of the critical thinking they develop.

Steve Pearlman: No critical thinking. That’s interesting. That was measurable.

Dave Carillo: That’s very

Steve Pearlman: Interesting. And this brings us back to our theme of the day, which again, is talking about how critical it is to have an assessment of critical thinking and that that assessment is in fact capable of driving critical thinking outcomes.

Dave Carillo: Yeah, use that assessment to your advantage. Let’s not make it. And peace. Let’s see what we can do to bring it into the rest of the classroom.

Steve Pearlman: And there are ample studies there’s even from back in Washington state and others that show that if we are able to offer direct instruction and critical thinking, however, that’s going to be defined in that assessment. But if we are able to directly instruct it, we can in fact move the needle, right? We’re seeing those same results at University of St. Joseph. We’re also very fortunate in being able to have been at this for a while and having been able to devote ourselves to this as a as a department.

Dave Carillo: Well, like we said before, if we can act as a bridge so that our listeners don’t have to spend the seven years holed up in a single office thinking about this constantly, then you know, let’s let’s be that bridge.

Steve Pearlman: Let’s be a bridge because nobody really

Dave Carillo: Wants to do that. Exactly. Holed up in the same office for seven years, especially with us,

Steve Pearlman: There are worse people

Dave Carillo: Like none come to mind, but there are probably a couple bad

Steve Pearlman: People, a couple of bad people. So let’s talk now about the importance of making assessment, not just assessment.

Dave Carillo: Yeah, well, that’s that would be the next step.

Steve Pearlman: That’s our next point. And this is what we mean when we’re talking about assessment. It can’t just be something that happens after the fact of the student’s effort. It has to be something that’s not only introduced to students early on in the process. It’s something that students have to be able to understand and maybe integrate into practice as they go forward. And that’s and then ultimately also be assessed by it, right? But that’s something that we stumbled on or stumbled over, I guess really early in our efforts here. So we here at USDA, we we did start to create an assessment tool. But what we realized more and more and more is that the instrument for assessing critical thinking and defining critical thinking is nice. What we’ve done over time is actually shrink the instrument and expand the language around the instrument to make it more functional for faculty as a pedagogy and more functional for students as a method so that it’s not just something by which they’re assessed. And we’ve seen so many times that we have faculty who simply take an instrument, you know, they just throw it in front of the students and say, this is how you’re going to be assessed, and that’s all they do. And that’s all

Dave Carillo: They do, right? And they didn’t attach that to a lecture kind of process.

Steve Pearlman: The students don’t make any gains.

Dave Carillo: How could they? Yeah, no. And I’m glad you brought that up because that was a huge stumbling block. We started with assessment and we were thinking about assessment, and we realized that it couldn’t just be assessment. There had to be a solid, authentic process involved. And that was and that was something that we had to learn for ourselves before we could help our colleagues figure that out as well.

Steve Pearlman: And I’m amazed at how at least, you know, I’m speaking personally, how remarkably naive a notion that was that if we could, if we could just define critical thinking, well, that that would be enough, that they would magically transform their abilities around that. And so in retrospect, that was a ridiculous premise.

Dave Carillo: Yeah. And I wish we had some language around that. We’ll have to maybe pull some old documents. But yeah, I mean, we we did a great job defining critical thinking with loads of wordy paragraphs and passages, but that didn’t really help the students at all. I mean, we could assess more or less whether they were they were writing and how they were thinking based on those lengthy paragraphs and passages. It was not easy to then go back and talk to them about rewriting, rethinking via that process.

Steve Pearlman: And so much of our work now really has been moving on from that point,

Dave Carillo: And it’s tough to really articulate. That’s that tricky sort of balance is one of the biggest pieces to this puzzle and to this podcast, I think, is this idea that the assessment has to be processed.

Steve Pearlman: I guess that’s one of our central caveats to all. This is that as much as we do want to talk about the important power of assessment, we can’t use that term as only something that happens at the very end has to be something that’s integrated into the educational system.

Dave Carillo: So, for example, and we’ll go back to go back to this idea of process, you know, you can say, Hey, everyone, you’re great is going to be determined based on your ability to consider different perspectives. That said, if you’re not doing work with them throughout the semester to consider different perspectives, then you’re not going to get that.

Steve Pearlman: However, a listener wants to define critical thinking and it must be defined, must also find ways to integrate that. And yes, that sort of brings us to where we came to with respect to the importance of our fifth point, which is making sure that that conception of critical thinking has as much of a naturalistic element to that as possible. And by that, we mean reflects the thinking process that students do, rather than imposing a thinking process or a conception of thinking that’s artificial and our. Conception of that, and we’ll just run through it really briefly, and, you know, there are others too, but our conception of that is that we start with how well does the student understand the issue and the source material around it? What problem question, concern conflict? Can the student pull out of that issue to discuss in their work? How does the student therefore evaluate the different pieces of evidence that are involved in that? What extent can students recognize the complexity of their own ideas, where their gray areas, where there might be things that they wish they could understand better, where there might be certain assumptions or biases or interpretations being made? And how do they draw conclusions along the way? And how do they use subordinate conclusions to build into larger conclusions? And finally, are they logical by connecting everything back to evidence? So our conception of that is essentially looking at some issue and weighing out the evidence to be able to draw a conclusion about it? And those five, those six elements to it, the understanding to find the problem. Do the evaluation. Recognize the complexity? Draw your conclusions. Include your conclusions and check your logic. Are central, we think to what everyone does when they’re confronted with any situation that requires some kind of conclusion to be drawn.

Dave Carillo: And it helps when we walk students through a couple of very easy sort of basic problems and illuminate those elements. They see that that’s basically what they are doing. You know, whether it has to do with what they’re going to wear in the morning or whether to bring an umbrella, you know, breaking up with boyfriend or girlfriend. Exactly. So they can do it for all those things we tell them, Yeah, you can definitely do it with a source or five sources or 10 sources. However, many

Steve Pearlman: Sources, right? And the trick is really just getting students to recognize and become self-aware of exactly those elements in the in their thinking and being able to bring that out into discussion.

Dave Carillo: And I think that that awareness, that metacognitive element to it is important because again, and we’ve said this, you know, before we think we think our process, we think our critical engagement is really important and helpful and useful. But you know, you use what you want to use. But making the students aware of the thought process in relation to how you’re going to assess them is huge. You know, that’s where that balance between assessment and process becomes really key. If you want them to do something that might not be as natural as some of the other things that we’re seeing that most people do when they think just bring that to the fore, say, here’s here’s what we’re going to value here, and this is going to be valued in terms of your grade. But let’s work through it,

Steve Pearlman: You know, and it should be something whatever conception of critical thinking you want to employ, it should be something that can function at any level. So we know that students do this and we’re working with people in middle school and working down to the elementary school level. More simplistic versions of the same thing because it’s what students do, regardless of whether or not they’re being assigned. It’s something that’s natural. It’s not a perfect representation. We don’t mean that of the cognitive model that humans used to think about things. It’s far from perfect. Certainly, maybe our listeners will have better ideas on that. But whatever it is, it should be something that’s universal in that respect, across disciplines and across grade levels.

Dave Carillo: You’re right, there are bound to be listeners out there like a much better concept of what’s going on in the human mind. But what we found when we did this work was that we needed to develop an assessment that allowed for the kind of complexity that we want from students. And they’re thinking about source material, but was simple enough that allowed them to understand that there was a process they were going through and for them to recognize when parts of that process were a little weak and needed to be strengthened, where parts of that process were strong and needed and needed to needed to be expanded and so on and so forth. So that was kind of a line that we walked

Steve Pearlman: When we’re still walking. And if there’s a central, I think if there’s a central challenge that you’re hitting on here, that’s central to the entire challenge of contending with critical thinking and education. It’s finding a way to conceptualize it that simple enough to be understood, learned and cultivated, but at the same time represents the complexity, the full complexity of critical thinking. I think we realized how that’s the balance that needs to be struck, and it’s a terribly hard balancing. We’re still, you know, we’re always going to be struggling to refine that.

Dave Carillo: Oh, yeah, absolutely. And we’re always looking at the work we’ve done to see how we can

Steve Pearlman: Improve it, and we are certainly our own worst critics. Yeah, although maybe that will change now. No.

Dave Carillo: Hey, critics. Well, yes, exactly. We might find out that we’re not nearly as bad as critics as we thought because they’re bigger ones out there. I don’t know.

Voiceover: This episode of the Critical Thinking Initiative is brought to you by the Critical Thinking Initiative. Org Visit the critical thinking initiative for an ever expanding resource of books, syllabi, videos and other resources on critical thinking. The Critical Thinking Initiative is always seeking out other people in education who want to work towards raising critical thinking outcomes locally and nationally.

Steve Pearlman: All right, so I guess it’s time now to move in to our question of the week. Let’s do it, and this is our very first question of the week. We’ve never had one before.

Dave Carillo: We’re super exciting. Like, like you said, we’re really well. We’re surprised and happy with with all the responses we got. So keep sending your questions and responses to us at info at the critical thinking initiative, dawg. But we did pick one to answer.

Steve Pearlman: So, yeah, maybe some lucky viewer next time will be the question of the week.

Dave Carillo: Yeah. Or, you know, we have a couple of good questions. We can answer two questions.

Steve Pearlman: This comes from Lisa in Virginia.

Dave Carillo: Hi, Lisa. You know who you are,

Steve Pearlman: And I’m reading from Lisa’s email here quote. I like the eco system concept, but with a four fold load in, about 80 students will having to teach critical thinking. Increase my workload? I agree. It is very important. I just don’t know how to even start without doubling the time I already spend end quote. And so that’s a very realistic question. Fair point. It’s a totally fair. Anyone who’s in education knows how fair that is to ask us, and we’re fortunate to say that the answer is that although there’s always a little bit of a learning curve, right, that you do not have to increase your workload at all, that it’s one thing to think about critical thinking as something else to be taught in addition to the content and in addition to the assignments and so on that are already on the books. And it’s a different thing to conceptualize critical thinking as the gateway to exploring all of that content and subject matter in the course. And so critical thinking can be integrated as the means by which those things are explored, rather than as something additional that has to be dealt with separately.

Dave Carillo: So in practical terms, think about things like front, you know, introduce your assessment or introduce the critical thinking process by which you want students to engage the material very early in the semester. You work with them the first couple of times they go through the process, work with them on the first reading in terms of having them work through that reading. Or you might have to devote maybe a little bit more time early on. But once students start to understand the process and understand that you’re assessing them on the process and understand how to do that process, you can kind of let them run wild. And so maybe you do a little bit more work with the first reading, but then the next two readings that are on their own. And so, like you said, it becomes a gateway, so you’re not necessarily in a position to have to double your workload if you can design, especially the first couple of weeks such that you’re introducing that process early on so that then you can let them go and do it themselves for the rest of the semester.

Steve Pearlman: Sure, so early on. Also, you can model and work through it, lead that process for the students and say, we have these two readings. Let’s talk about the relative value of these two readings together. And then as you start to pull your hands off, you can actually put more onus on the students. Yeah, and rather, in fact, than taking time away from your teaching and in fact, save you some time because we’re able to put onus for understanding and reasoning more and more on the students, which actually take some of the burden off of us as educators. It’s not a magic trick. It doesn’t happen right away. There’s no rabbit that’s going to come out of the hat, but it is something that happens over time over the course of a semester.

Dave Carillo: And what we like to talk to faculty about is is that there’s going to be a certain amount of hand-holding early on, but just make the students aware that you’re holding their hand now so that you can let go them later. And if you work closely with them early on, they’ll they’ll be able to to do more on their own. And so, like Steve said, you actually end up saving time when you can give them one or two readings and say, OK, you know what to do. And at that point, too, you’ll find that the students are more engaged anyhow, so you don’t necessarily have to find ways to enter into the material for that week or that unit or so on and so forth, because the student you’ve given the students a way to enter the material themselves.

Steve Pearlman: Lisa, we hope that’s a good answer, and you should email us back to let us know how we did.

Dave Carillo: Yeah, exactly. We hope that helps. But thanks so much for the question.

Steve Pearlman: And so that’s going to take us into our news of the week.

Dave Carillo: These are the week Dave

Steve Pearlman: I went first last time. Did you means you have to go first this time?

Dave Carillo: All right. Ok. I will go first. I’m grabbing the article now that I that I printed out the other day, Steve. This comes from Peoria, the Peoria Public Radio. Org That’s Peoria. Main, I believe not. Peoria is near a Peoria Washington, too, or I think there are a number of pure number. Is there? Yeah, maybe. Yeah. Well, this one is this is this is me. This article is called Do Laptops Help learning a look at the only statewide school laptop program. So like the title suggests, Maine has the only statewide laptop program. I didn’t know that. Yeah, neither did I until I read this, whereby every public school student gets a laptop and they have aimed. Over the last many years, 12 years, I think maybe even more. It started in two thousand so more. The aim was to to move the kids toward a kind of classroom experience that revolved around the technology that was more or less paperless and allowed students an easier way to get that information and get at different kinds of information and so on and so forth. But people are starting to question whether that’s actually working, and one of the individuals quoted in this in this article is his name is James Welch. He teaches an American politics class at a high school. And the article goes on to talk about his experience quote Well, first of all, James Welch says that when he arrived in Gorham High School seven years ago, he’d never seen so many computers in one classroom.

Dave Carillo: Well says it turned the class into an interactive discussion, which sounds pretty good. That sounds great. He says. It’s like we can put the world on the desk of each kid. His students write blog posts, read each other’s work and share videos and articles all online. So so far, so good, right? Great programs accomplishing essentially what it was intended to accomplish. That said, Mr. Welch goes on to say that he starts started to notice that when students turned in some of their essays, their writing wasn’t as fluid as it was when students were putting pen to paper. He also said quote You could also see an increase in copy and paste, whether it’s from another student, whether it’s from a piece online, digital sharing is what these guys do. End quote. So now he requires his students to write out their essays by hand, which is interesting. But the point was, and what I wanted to get to is that James Welch and I’m maybe we can assume other individuals are starting to notice that you can’t just put. And this is also a quote from the article. You can’t just put a computer in a kid’s hand and expect it to change learning.

Dave Carillo: Hmm. Right now, the article goes on to say that research, although it doesn’t say which research research, has shown that the one to one programs meaning one student, one computer implemented in the right way increased student learning in subjects like math and science. And those results have prompted other states like Utah, Utah and Nevada to look at implementing similar programs. But it stands that again, Maine is not seeing large scale increases in student learning now that they don’t define what student learning is. And for our purposes, though, I brought this article in because this is this illustrates a really, really big point for us, right? That it’s not necessarily the technology that the students have, and it’s not necessarily the materials that the students have. There has to be a process in place, especially for critical thinking, and that process has to encompass encompass things like assessment, and it has to encompass things like pedagogy and process and so on. So it’s not the computer that’s going to make these kids smarter. It could help them, obviously, and can help them find sources to engage, and it can help them find information to evaluate. But in the end, it’s turning out that it’s it’s not going to be the computer that that helps these students through. It’s going to be the pedagogy, it’s going to be the process.

Steve Pearlman: This is an exciting article because it’s such a large scale program in Maine. Yeah, and we’ve seen some other research on this, and we know that computers can be yet another valuable tool to learning. But they’re not the difference maker really, when it comes to outcomes, and they’re certainly not the difference maker when it comes to critical thinking. So. But this really shows that on a scale that I think is important. It’s not necessarily that there are obstructions, but the real differences are going to come through other tools and instruments and processes and pedagogy.

Dave Carillo: Absolutely. And you know, it’s you know, it’s maybe we’ll be able to visit this at the same time. But you know, the research is still out on on the more modern versions of this like E! Portfolios, too, that people are starting to use. There’s not necessarily a lot of research saying that those are facilitating critical thinking any better than not E portfolios as it were.

Steve Pearlman: Right? And every time a new technology emerges.

Dave Carillo: Right? Yeah, there’s always

Steve Pearlman: So many people rush to it as the solution, and it’s never going to be. I don’t think they’re ever going to come up with the technology that is going to make people think better. It’s going to have to always come down to some human interaction with somebody who can help you generate and refine your capacity,

Dave Carillo: If anything, now, right? You know, having your hands on so much information is starting to cause some problems with students who aren’t able to evaluate the strength of that material. So there you have it. There’s my article.

Steve Pearlman: So let me switch to my article here, which is by Laura Silver, and it’s four charts on how people around the world see education. And this is from the Pew Research Center, so you can see it at Pew Research. And what it did is it looked at some of the more advanced countries in the world and also looks at emerging and developing countries like China, India, South Africa, Nigeria and Kenya. But this is classifying advanced countries as those that are, I guess, more progressed in terms of their overall development, and it’s looking at what’s the best type of education. Ok. And they are two major metrics that looked at. It should said, should education either focus on quote basic academic skills and encourage discipline, end quote, or should it focus on helping students quote to be creative and think independently end quote. So it’s either do we want basic skills and knowledge and discipline or do we want to think creatively and independently? Ok? Among the countries that thought that education should focus on basic skills and encourage disciplines, the United States was fairly high up on the list. So the scores ranged from a low of twenty four percent, which was Spain, where the fewest people thought that that was a valuable part of education to a high of fifty one percent, which we saw in the UK and in France. So fairly conservative views about education actually from UK and France, which is a little surprising to me. And the U.S. was a 42, which was higher on the scale. We’re not the top, but we’re fairly high in terms of whether or not we should teach students to be creative and think independently.

Steve Pearlman: U.s. was roughly in the middle, so we had the highest people in Spain again, and the lowest was in the UK was the most conservative. Another part of the study that was really interesting was actually looking at the views of younger people in these countries and seeing the extent to which younger people felt differently about this. And this is really the point that I want to get to the most here. Yeah. Quote younger people in most advanced economies are the most supportive of education that emphasizes creative and independent thinking. The difference is largest in France, where fifty three percent of those ages 18 to thirty four say creative education is more important than teaching skills, compared with twenty nine percent of those 50 and older in the U.S.. The difference between these same groups is 17 points, really just 17 points in the U.S.. Wow. So surprisingly, our younger generation is not as, I guess, supportive of independent thinking and creative thinking as part of their education. And that, to me, is actually a very distressing metric for us and speaks to the importance of bringing this school thinking into education across the country. Because it’s not only that our country as a whole isn’t at the forefront of valuing this in terms of how the populace sees the role of education, but even also that the younger generation which needs it more than any of us, especially with respect to the emerging needs of the modern workforce, is not seeing the value of it. And to me, that’s something that is a call to action

Dave Carillo: That is really interesting. And I think there are a lot of interesting elements at play, and I agree we need to bring the the importance of critical thinking to the forefront. And I wonder it hasn’t always been this way, but it becomes sort of a traditional reason. You know, why folks go to college is so that they can get a job right. So I can see where if students are misconstruing sort of basic skills, you know, as the way to get a job, then I could understand why they might feel that way. But that kind of thinking is causing a lot of tension, like you said, with the fact that kinds of studies that we’re seeing coming out of workforces that if anything would be valued more than basic skills more than even specific majors or grades, is this idea that individual needs to be able to sell problems, an individual needs to be able to think critically. So that’s really interesting.

Steve Pearlman: I like the point you’re making here a lot concerning that relationship or the tension that’s emerging. And I think something else about this that I struggle with is that, I mean, I’m not blaming the Pew study for this at all. The Pew study is illuminating this, but it’s the it’s this perception in the country that not if there’s this either or between the two things, yeah, right, but that at least the two shouldn’t coexist. And one of the things that we found a lot is in terms of basic skills and disciplinary things and so on. A lot of that easily comes along for the ride when we’re able to get students invested in genuine intellectual work in the classroom. Acquisition of basic skills are far less of an issue to have to contend with that if we move with the higher order skills, the lower order skills largely obviously there are exceptions and their classes have their own personalities and so on populations. But if we move with higher order skills and we’re able to get students really invested. And higher order work, the lower order things come along for the ride.

Dave Carillo: Agreed. I mean, it echoes it echoes that critical thinking versus content question that we hear a lot from faculty who generally wants to teach critical thinking, but are either under this great amount of pressure to push content or are very hesitant to quote. Get rid of that content for critical thinking. And you’re right, it’s it’s it’s it’s about bringing those two things together and moving forward with the higher order skills. Like we said before, as the window through which content is viewed, you know, is engaged.

Steve Pearlman: Yeah, and it’s a shame. It’s it’s a shame that and again, I like the pew question, so I’m glad it illuminates this. But this is a false dichotomy. And we have you don’t have we don’t have to choose between these two things. We have to

Dave Carillo: Move beyond the either or. Exactly. In a lot of arenas in our lives today. So hopefully we can do that at some point.

Steve Pearlman: So thanks again for listening to the Critical Thinking podcast. I think that wraps us up for today. Yeah. And we’ll try to have a another podcast out to you in a week or so. Absolutely.

Dave Carillo: Everyone know. Keep, keep your comments coming. Keep downloading and keep sharing with your friends. And again, if you have any comments or questions, you can get in touch with us at info at the Critical Thinking Initiative. Org. Thanks so much.

Voiceover: Thank you for listening to the Critical Thinking Initiative podcast. Got questions for Steve and Dave? Just send them to info at the Critical Thinking Initiative dot org. Follow us on Facebook and Twitter and find us at WW W Dot The Critical Thinking Initiative. Org.


Submit a Comment