Does Higher Education Improve Critical Thinking?

PUBLISHED: Jan 7, 2020

In This Episode.

Steve and Dave delve into recent research on critical thinking growth throughout college. Learn the extent to which it is happening and why. Warning: this episode may contain some ranting.

Episode Archive

Podcast Transcript

Voiceover: Welcome to the Critical Thinking Initiative podcast, we bring you research driven solutions to critical thinking education. Why? Because as Bertrand Russell said, most people would sooner die than think. In fact, they do so. And now your hosts Steve Perlman and Dave Carillo, everyone.

Steve Pearlman: Welcome back to the critical thinking initiative. This is Steve Pearlman.

Dave Carillo: I’m Dave

Steve Pearlman: Carillo. Before we get into our podcast today, we want to encourage you to please check out our new podcast, Smarter because our podcast won’t just make you smarter. It will make you smarter.

Dave Carillo: At least we’re claiming that. I mean, don’t don’t take us to court

Steve Pearlman: This new podcast is for everyone. While the Critical Thinking Initiative podcast focuses on education, and we hope you will please like it and share it and give it ratings on iTunes and recommend it to everyone you know who’s an educator or an education enthusiast. Smarter comes in shorter episodes with tips on how to focus on critical thinking in our daily lives, in leadership, in business and parenting. It’s offering little tidbits on how the brain works, and you get to hear Dave and I exercise some of our favorite movie clips as references for the critical thinking acts. What could be better than that, Dave?

Dave Carillo: Almost anything,

Steve Pearlman: Actually. Thank you for supporting the podcast.

Dave Carillo: As you can see, I value our work highly, but yeah, we realized that our listeners will probably at some point during the day, leave the classroom and go home, and there’s probably other folks that might want to use some critical thinking research to their advantage.

Steve Pearlman: What happened was people contacted us and let us know that there are people in the world who aren’t in education and yet also have an interest in thinking. And this was new information for us.

Dave Carillo: So we did some empirical research. And lo and behold, there’s a whole world out there with like stores and cars and billboards and movies and things. It’s it’s crazy.

Steve Pearlman: We had no idea. So please check out smarter. We think you’ll really enjoy it. Please recommend smarter to anyone you know who wants fun, helpful tips for how to think critically in daily life. How to understand how their brain works, how their friends brains works, how their kid’s brain works and so on. And it really should apply to just about everything everyone does in their lives. But for today’s podcast, we are a little confounded about how to approach this topic. So what you might hear from us today is some ranting. It will be research informed ranting. We always stick to that for which we have the best evidence. But I think in a nutshell, we want to bring some greater awareness to the gravity of the critical thinking problem in education. And there have been a couple of studies released of late that start to talk about this more. But for us, having been in the trenches on this for a long time, we think a greater call to action needs to be made and that there’s simply not enough attention being paid to this problem. And even though we are glad to see these new studies starting to take form, it’s a drop in the bucket compared to what needs to be done. And so we’re hoping today’s podcast will inform you and arm you with a little bit more knowledge and information about the gravity of this issue and the complexity.

Steve Pearlman: And our hopes for that is that it motivates your teaching and motivates how you interact with your colleagues and start to bring their attention to this. It motivates what you go and do at your institution, because simply put, there are probably a few things education needs to do at the pinnacle levels of importance. And one of them is probably developing character, which is not in any way mutually exclusive to developing critical thinking. And another is developing compassion, which is also not mutually exclusive to critical thinking. In fact, one can amplify the other as a matter of fact. But finally, it’s critical thinking and the world is looking to education to accomplish this, and the world certainly needs it. Given the problems that we’re facing and all of our greatest advancements in history have been a result of people who could think about the fundamental level, we’re not really teaching students to think more critically. Then it becomes pretty hard for us to defend the premise of education, no matter what knowledge students have. If they can’t think about it, if they can make better decisions, if we can advance the world, then that fundamental root we are failing. End of Steve’s rant. Thank you for hearing me out, right?

Dave Carillo: Let me just temper that by saying, Yeah, I agree with what Steve saying. What I find interesting about the research that we have convened is that there are aspects of this research that will, like Steve said, communicate what’s at stake overall. And it’s a really complex subject in certain sense, and we’re really just giving you the tip of the iceberg. But I do think that Steve’s irate

Steve Pearlman: Rant aside, damn right. It’s irate, it

Dave Carillo: Is a little irate. I’m irate to what we’re talking about overall is just the continual development of a human brain to meaningfully engage in a complex world, and that goes well above any base of knowledge, right? But let’s get to it.

Steve Pearlman: Here’s the nutshell that needs to be communicated at the outset. And it is what’s framing this entire discussion and where my rant came from and what the research is once again reiterating to us overall, if there are gains in critical thinking at all at the university level, as Derek Bok said, he was the former president of Harvard and he said it in his book, The Title of which I don’t remember. But he basically said, Look, if there are gains happening in critical thinking in colleges, they are at best modest and that’s what’s prompting this. The research that’s coming out is showing us that potentially there are some gains for students in critical thinking in terms of their college experience and potentially not. One of the big confounding variables is simply that there is no control group relative to students who haven’t gone to college, so it’s been very hard to exclude the contribution of human development as we move into adulthood relative to actual training in college. Best case scenario, though, is that if it does happen, it’s it’s a modest thing. Worst case scenario is that it doesn’t happen much at all. And what we’re going to talk to you today about, as well as that, there’s some evidence that if it is happening, it’s happening in the earliest stages of college, first and second year and then declining in terms of its rate of growth or stagnating in terms of its rate of growth for the third and fourth years.

Steve Pearlman: And the reasons for that are speculative relative to even what researchers are examining. It might be the nature of Gen Ed courses relative to courses in the major. It may be that as students move deeper into the major, the emphasis on a content push become stronger rather than on the acquisition of skill sets. We’re just not sure. And again, the research is not conclusive on any of this. What is conclusive as far as we can see, is that if there are gains and again, this is the point as well, as well as many other researchers who have examined this. If there are gains over the course of college, they’re humble ones, and that’s simply not good enough. And that’s where we want to get into what the research is disclosing and some of the reasons and challenges around that. So we can all collectively try to face this problem and stand up to it as educators.

Dave Carillo: With that, I want to just briefly discuss the article that we both kind of came to that started us thinking about this in terms of a podcast episode, and that’s entitled Critical Thinking in College Freshmen. The Impact of Secondary and Higher Education. And this is by Murray Evans on Furberg and John Ellen. They’re out of the Center for Instructional Psychology and Technology and live in Belgium. This is from two thousand thirteen, and this is actually the oldest article I think we have today in front of us. But Steve, like you say, they mentioned in their own review that most researchers find that critical thinking grows primarily in the first two years before it levels off or even drops out, and they cite a bunch of different individuals all the way back to, I think, nineteen ninety two. But this is an interesting study, specifically because they wanted to measure the growth of critical thinking in college students in their first year and to start to figure out whether certain elements from secondary education or their high school education could explain how students make critical thinking gains. They were looking at differences in the entrance performance based on secondary education. Are there differences in growth based on secondary education and higher education? And are there differences in performance based on secondary education? They say.

Dave Carillo: Firstly, quote, the results of this study indicate that growth in critical thinking during the first year of higher education is on average small. They note that if you rolled into college from secondary education or high school educational background, that focused more on a sort of general education, which is an equal mix of science and humanities and arts and math and so on. Those students basically showed larger growth in critical thinking during the first year. And finally, that quote students in a professional bachelor program grow more in critical thinking during the first year of higher education than students in academic bachelor program. So there’s two takeaways from this. They found that critical thinking growth during the first year of higher education is on average small, and they also found that students in a professional bachelor program grew more critical thinking than those in academic bachelor program. Which might seem counterintuitive because the professional bachelor program seemed to be focused more on real world vocational professional types of skills and the academics. One would think we’re focusing on more of those academic things that would foster critical thinking, but they didn’t find that what

Steve Pearlman: They’re saying is so important. And again, this is a Flemish example, and it’s only one study, but it’s one of the only studies that’s really looked at this the people in the professional programs such as vocational and secondary, and I’m not sure exactly how that translates out are showing the most gains now. We can only speculate as to why, and the researchers themselves say that they don’t know exactly why, but I think there’s some indictment there of what’s quote unquote the purely academic. And that is so much of the education that we. Often see, and it contributes to the reason that academia gets criticized as this sort of displaced ivory tower, whereas people in these professional programs who might potentially be engaging in things that are more real world problem solving, problem based learning, how do you actually make something? What you actually do in this situation are potentially showing the greater gains. Some of this is a little speculative. I want to be careful that I’m not creating the answer where we are unsure, but the result is nevertheless that we’re seeing that these professional students are doing a little better

Dave Carillo: And even if they’re doing a little better. What this study found overall is that the gains are on average small. And I think while this study is unwilling and rightfully so, to say for sure why that is after we go over to the results of these other studies, I think there are a few different moments in all of them that start to provide us with stronger answers as to why that is.

Steve Pearlman: I think perhaps one of the biggest studies coming out in recent years, and credit goes to them for their efforts on this is by Huber and Kunsthalle, who wrote an article called Does College Teach Critical Thinking? A Meta Analysis. They came out with over one hundred searches that they whittled down to seventy one, that they really looked at, and there are some challenges with how they whittled. I think, and I understand their need to do this, but one of the things that they did was they changed a search for critical thinking to changes in critical thinking, which may predispose a kind of selection bias around studies that actually show changes as opposed to studies that didn’t or specifically talk about that. But it’s an understandable need to narrow the field.

Dave Carillo: It’s not like the other reviews in these studies that we saw are substandard in any way, but they do a good job to sketch out the basic conversation going on around this, and they do a nice job of at least giving the overview of where the conversation is.

Steve Pearlman: Another thing they do that’s also important. They make a differentiation between cross-sectional studies, which might look at students who are freshmen and then students who are juniors within the same institution and longitudinal studies that track the exact same students across multiple years. And this is very important. The cross-sectional studies produced larger effect sizes than the longitudinal studies, and they note some pros and cons to that. They suggest that the cross-sectional studies might favor those students who tend to stay in college. And again, they’re not able, and they acknowledge that they’re not able to eliminate the confounding variable of simply what occurs in terms of critical thinking through developmental growth. At least some people mature. I guess in becoming an adult, I’m waiting for that to happen to me. But what they found was an estimate of a zero point five nine standard deviation movement over the course of college through these critical thinking studies across the meta analysis. This is where this becomes particularly salient and important for our purposes. By their accounting quote, it is worth noting that a zero point five standard deviation gain for a person who starts at the 50th percentile would lift him or her to the sixty ninth percentile. No small improvement in our minds and quote Well, here’s the thing that’s an awfully small improvement in my mind. If over four years of college, four years of college, we were moving someone from the 50th percentile to the sixty ninth percentile, and that’s what we’re claiming as a substantial gain, then I think we have identified the deeper nature of the problem.

Steve Pearlman: Imagine that if we’re doing that for people in all their subject areas, and if we’re grading those people purely on a curve relative to the performance of others, we’ve moved someone from an F to a D plus over the course of college. Now that’s not against a literal standard, necessarily, that’s against a curved standard relative to other performers. But we have to stop thinking about knowledge as being the valuable commodity of education. Knowledge is now accessible to everybody. Everybody can learn broad swaths of knowledge thanks to the internet. Sure, some of the most rigorous academic work is withheld in terms of its culture, in terms of its language and in terms of its access from the general public. But knowledge is no longer the big commodity it’s thinking. And if that’s the primary thing that we’re hoping that higher education accomplishes, and its movement represents one from the fiftieth to sixty ninth percentile or equivalent in other ranges over the course of an entire college career. That, to me, is not good news. That, to me, is an indictment of what we’re doing in education. And that’s, by the way, still, again, one of the more hopeful outcomes that we’re seeing from the research.

Dave Carillo: So, yeah, and I just want to go back to some of their discussion to Steve because what they say right off the bat is that quote. Our study suggests that students make substantial gains in critical thinking during college end quote, and then they go on to discuss that standard deviation, which they say quote paints a slightly more optimistic picture than recent estimates by Aramin, ROSCA and Pascrell and Taryn Zinni. And, of course, Aramin. Orozco is academically adrift, which is probably the most well known of these kinds of studies. But Pascrell and Tarantino have done two big studies I think one in two thousand five and another one in 2011, I think, and both of those studies and academically adrift showed minimal gains. Throughout college for critical thinking, and as such, they start off saying, well, our study shows that students are doing this more than those other studies, but then they follow that up with quote. However, our overall findings are fairly consistent with these studies. Aramin ROSCA estimated gain of zero point one eight standard deviations over three semesters, which falls well within our 80 percent credibility intervals. Similarly, gorilla and trendiness overall estimate of zero point five zero standard deviations from a combination of longitudinal and cross sectional studies is reasonably similar to our mixed design estimates of zero point five nine standard deviation. So while they make the claim that they’re seeing substantial gains gains that are more impressive or more positive than this other data, they go on to say almost immediately that their overall findings are in line with that data, so they’re not seeing anything better than anyone else who’s saying that critical thinking outcomes are modest, if at all present, and Huberman can still go on to say this quote.

Dave Carillo: Another somewhat worrisome finding is that observed gains in critical thinking appear to have deteriorated over time, despite increased interest in fostering critical thinking skills. End quote. There are two other studies here I’m going to make quick work of them in order to further contextualize what’s at stake. This one is a student learning and higher education, a longitudinal analysis and faculty discussion. This is by Mather’s Finney and half from two thousand and eighteen. Number one, they found that on average, students had moderate gains after experiencing one point five years of college, so they didn’t find massive gains. They didn’t find no gains, but what they found was moderate. And the second hypothesis that gains will increase with increased coursework ran contrary to their expectations. So they said that unfiltered gain scores did not increase with each additional course completed in the domain or gain scores increased after students completed one quantitative and scientific reasoning course, but then leveled off after multiple courses were completed.

Steve Pearlman: So let’s emphasize that point a little bit. Two things First of all, the course that was explicitly teaching reasoning did help, and then that skill atrophied over time when they just got into other courses that weren’t focusing on it right? And more college learning more classes in the discipline did nothing to foster additional critical thinking growth, and this is the very concern that we’re touching on for this podcast. We want to turn this conversation to education to critical thinking first and foremost, beyond content, beyond all else, because it is the thing that we need to push

Dave Carillo: Us forward right in this last study that I want to touch on entitled holistically assessing critical thinking and written communication learning outcomes with direct and indirect measures. They looked at the critical thinking and written communication outcomes between first year students and senior students in a single college within their larger university. They call that college a, and they study their students using the ETS heightened written communication assessment and the ETS heightened critical thinking assessment, which I don’t want to go into. But they describe them as mostly multiple choice. And then they also used a self-efficacy survey from the National Survey of Student Engagement. Essentially, they found some evidence that students are developing effective, written and communication skills between first year and senior year. But for critical thinking, they found mixed evidence. They found that generally the higher critical thinking scores seem to correlate with the higher GPAs. But overall, they found that senior scores increased, but not significantly compared to first year students. So they found those same kinds of things that these other longitudinal studies are finding no significant gains,

Steve Pearlman: And I want to piggyback on that with another study. This one is by Lou Lumo, Frankel and Ju. Assessing critical thinking in higher education, the heightened approach and preliminary validity evidence. And this found some similar things. But here’s the thing I really want to bring out about what this study discovered is a glaringly bright signal that’s flashing at us about how we need to approach critical thinking and the importance of that. And this study looked at freshmen and I looked at seniors, and it made some assessments of the difference in their skills with respect to critical thinking. But here’s what it also writes quote about seventy one percent of the test takers indicated that they use their best efforts in test taking in the rest admitted that they didn’t try their best. The performance gap between these two groups was alarmingly large at zero point seven standard deviations. To put that result in a context, the performance difference between freshmen and seniors on average was zero point three standard deviations, which suggests that the score difference introduced by motivation is more than twice as large as the difference possibly introduced by four years of college education. While end quote and they say that this is not unique to the study, there are other studies that have shown motivation as a stronger defining or as a least as a very powerful defining factor between outcomes on critical thinking tests than any other factor. Now. We have students who are simply doing better based on how motivated they are to take the test, more so than four years of college education. What are we doing in college education? And I want to be clear again, it’s not that students don’t need knowledge, it’s that there’s no mutual exclusivity between getting students knowledge and putting critical thinking about that knowledge as the primary thing that they need to learn first and foremost. But that doesn’t seem to be happening, and this is our wake up call as an educational community about how we need to change and where we need to go.

Dave Carillo: What was that question that you asked? Like, what are we doing in college, right? Right. I see some evidence as to what we are exactly doing in college. We can remove it from that sort of rhetorical irate sort of stance that we generally take, like, what are we doing in college by looking at some of the evidence specifically from these studies? And so I’m going to go back to that holistically assessing critical thinking and written communication study that looked at college a, they use the heightened tests for written communication and critical thinking, but they use the NSC to survey the seniors who took these exams. And when asked about the extent to which they thought their college education contributed to their ability to use critical thinking skills, what these seniors seem to report is that quote compared to students affiliated with other colleges at our institutions, Students in College A reported that their courses placed more emphasis on lower level skills like memorization and less emphasis on higher level skills, like analyzing an idea or forming new IDs. So the students are reporting that they see an emphasis more on lower level skills like memorization and less emphasis on higher level skills. Like I said earlier in that big meta analysis by Huber and Kunsthalle, they go on to say quote. Although there is little disagreement that critical thinking is important teaching it takes time away from teaching other important skills, such as reading and mathematics, end quote.

Steve Pearlman: Steve is pulling the hair out

Dave Carillo: Of his head, look right off the bat like reading to decode rather than reading as a critical meaning. Mating act at reading as in critical reading as in is in naturally involved in critical thinking. The way that reading and writing and singing are interrelated is a huge red flag to me.

Steve Pearlman: It sets up this false alternative, right? We can use critical thinking to read better, and we can use reading as a vehicle for also learning how to think critically. Any time that we are doing reading in our classroom, we can also be talking about critical thinking at the same time. And I want to quickly refer to another article. This is by Shimon Balzac. The impact of faculty teaching practices on the development of students critical thinking skills. And what’s interesting here is that they surveyed students about what they thought was helping them in terms of critical thinking. They also hit them with the AP test as a way to measure their critical thinking, and they looked at what students thought was achieving critical thinking. And B, whether or not they actually grew at all, at least in terms of this particular test. And the cap is a 40 minute thirty two item instrument designed to measure students ability to clarify, analyze, evaluate and extend arguments. It’s four passages in a variety of formats. Each pass is contained a series of arguments that support a general conclusion and a set of multiple choice test items. So again, we have another multiple choice thing here. Of all the things that they looked at, most were not associated with any growth and critical thinking. We’re talking about course related tasks such as writing, problem solving, class presentations, group projects, applications of concepts, compare and contrast, defending a point of view, integrating ideas, faculty feedback on their work.

Steve Pearlman: None of this was actually associated with growth on the GAAP. Some of it was associated, certainly with students self-report. Students felt that they were becoming better critical thinkers through these things, but nevertheless, even though students reported gains from a lot of these things, it didn’t show out in the test. The only thing that was correlated with a growth in the cap was asking challenging questions that was found to be representative of growth by 0.5 seven points, which was statistically significant. And interestingly, group projects was negatively associated with growth in critical thinking. But all of us is a long way of getting back to your point, which is simply this the mere presence of these acts in our classroom. These things that we think are fostering critical thinking, if not applied well, if not really understood in terms of their components. We’ve talked about this in other podcasts as to the intricacies of what makes those things valuable, right? But without those intricacies, all these things that many faculty members think we’re doing in our classes to foster critical thinking may not actually be doing that at all.

Dave Carillo: You’re right, and we have talked about this in other podcasts. There are ways that group projects can foster strong critical thinking outcomes, and there are ways that they can. There are ways that group discussion can, and they can’t. We can at least benchmark or draw a distinction between the kinds of things that are trying to achieve critical thinking and the kind of things that we know absolutely don’t memorization teaching to a test like. Sure, you hear those arguments left and right as to what those methods are not able to do for students in terms of critical thinking, yet we know from other research that most of the time that’s kind of what’s going on. A lot of lecture, a lot of tests. We have these students reporting that what’s valued is mostly memorization and low level skills, and this is my last point for the day. This is in the holistically assessing critical thinking and written communication study. This is where the students noted that memorization was valued, and I will say kudos to this school. Based on these findings, even though there was some vigorous debate, they did work hard to revise their courses. Now they don’t say how and when and where, but they say early on in this paper that institutionally we define critical thinking as including creative thinking, innovation, inquiry and analysis, evaluation and synthesis of information. Students will be able to demonstrate critical thinking skills through a process of inquiry that explores evidence for developing innovative and creative solutions to make informed decisions and evaluations, but to see some major problems with this OK. First of all, creative thinking and innovation are big subjects

Steve Pearlman: Very hard to assess, and we talked about that very hard to. We talked about that in our Labs podcast.

Dave Carillo: Right, exactly. Very hard to assess and not necessarily always linked with actual critical thinking. You can be highly creative and not have thought through any of the elements of that situation and that creative thinking may or may not be effective.

Steve Pearlman: Creativity is such a catchall phrase, right?

Dave Carillo: All we see here are terms not necessarily process and not necessarily a way to assess. And we know that they’re using the ETS assessment, which means that they’re assessing with a multiple choice test. And so even if we just table all the lecture and memorization and we just all agree that that’s not the way forward, even when we find curricula and assignments and definitions that intend to teach critical thinking or aim to teach critical thinking or claim to teach it, we see some major conceptual and pedagogical problems.

Steve Pearlman: And I guess that’s where we land, which is that the layers of the problem tease out troublingly, if we do see critical thinking gains, we’re troubled in many ways. By the way, they’re measured with these multiple choice tests very often, not always. And even then, they’re humble. Potentially, we don’t see gains at all. Potentially if we do see gains, they’re maximized in the earlier years of college when they should be continually developed throughout college and not stagnate. And even when we see faculty employing pedagogies that potentially lend themselves to critical thinking. Those are troubled because often the intricacies of those pedagogies are not employed within the practice. And that’s not necessarily a fault of the educators who are trying hard to break away from lecture and rote regurgitation into these other practices that are more constructivist and open the door to possibilities. But there are ways that these things work, and then there are ways that they don’t. And most of the research we’re seeing is that the mere adoption of these potential pedagogies, to your point, isn’t necessarily accomplishing the end, and that returns us again to where we started. That closes the circle. We have to make this the priority. And yeah, Dave and I feel as though we have made some strides in this and we have a system that, though incomplete, accomplishes some wonderful things.

Steve Pearlman: But this isn’t about us or about our system. And if the point here is that as an educational community, if we’re going to solve the problems that we need to contend with as a species, we need all education to focus on critical thinking foremost and stop the notion that the acquisition of knowledge is any longer. The most valuable commodity we have knowledge is growing faster than anyone can keep up with. There is no way in our field that we can read everything that’s published. Knowledge is advancing and being created faster that we can keep up with the skill to reason through that knowledge, to advance the human condition, to ask the interesting question and be able to approach that question and try to find some answers to it and to work collaboratively with other humans in that process. That’s the only thing that’s going to advance us forward, and that’s the thing more than anything else that education needs to focus on. And that’s what has us riled up today, is this continued information that it’s simply not

Dave Carillo: And that’s skimming the surface of the problem. As always, if you have any questions, reach out info at the critical thinking initiative, dawg. Thanks for listening.

Steve Pearlman: Have a great day. Have a great day.


Submit a Comment