Are You Teaching in a Critical Thinking Ecosystem?

PUBLISHED: Aug 23, 2017

In This Episode.

In this first episode of The Critical Thinking Initiative, Steve and Dave take you beyond the popular misconceptions of why critical thinking is so hard to teach.  Learn the five research-supported factors that undermine critical thinking outcomes.  Steve and Dave will help you start to understand how to build a critical thinking ecosystem in your class or at your institution.

Episode Archive

Podcast Transcript

Voiceover: Welcome to the Critical Thinking Initiative podcast, we bring you research driven solutions to critical thinking education. Why? Because there’s Bertrand, Russell said most people would sooner die than think. In fact, they do so. And now your hosts Steve Pearlman and Dave Carillo.

Steve Pearlman: All right. Welcome to the Critical Thinking Initiative podcast. You’ve got Steve Pearlman

Dave Carillo: And Dave Carillo,

Steve Pearlman: And we are launching this podcast in an effort to help stimulate the discussion among educators around the essential need to heighten critical thinking outcomes nationwide and a little background very quickly on the critical thinking initiative. Dave and I both work at the University of St Joseph, where I serve as the director of interdisciplinary writing and reasoning,

Dave Carillo: And I am the administrator for the Critical Writing and Reasoning Portfolio Program.

Steve Pearlman: We both sought the longest possible titles we could find because we do feel as though that makes us better experts. So the critical thinking initiative is borne of the fact that Dave and I have been fortunate enough really as a credit to University of St Joseph to have been able to essentially lock ourselves in a cave for the last six years and study in a dedicated fashion what actually produces strong critical thinking outcomes. And we’ve been tasked with changing the critical thinking outcomes and elevating critical thinking outcomes across the campus at USGA. And we’re excited about some of the things that we’re accomplishing there. And we feel it’s essential, given the state of affairs today in the world, that we share some of the insights to which we’ve come about critical thinking and how to teach critical thinking and assess it, because certainly there’s never been a more critical time and the achievement of our students with respect to critical thinking, in speaking about our nation as a whole and nationwide outcomes. It’s not something we’re exceptionally proud of.

Dave Carillo: No, we’re not necessarily seeing the kind of outcomes that I think most of us would like to see from our students. And you were hoping that this podcast does a little something to further that discussion and we’re hoping to to develop a relationship with our listeners and engage in an ongoing discourse about critical thinking, critical thinking, pedagogy in the classroom across the disciplines in an effort to help raise those outcomes, however we can.

Steve Pearlman: So in one sense, we hope to be a resource for our listeners. In another sense, we hope our listeners to be a resource for us and for other listeners. At the same time, I want this to be a discussion,

Dave Carillo: A rich dialogue.

Steve Pearlman: Absolutely. Now, just to refer back to what we were saying about some of the struggles we’re seeing with respect to critical thinking outcomes nationwide. Let’s be really specific about this, and we’re only going to give you one example today because we could go on about this and we don’t want to make this look too dark.

Dave Carillo: We don’t want a problem you out. Yeah. In the process. And you know, you might if you’re listening to this podcast, you might know this already. But if you don’t know it, this is going to open your eyes.

Steve Pearlman: There’s just no question that among employers, for example, and we certainly are not making the argument that critical thinking should only be developed for the sake of employment. Not at all. Not at all. But if we want to see the extent to which our society needs critical thinking. For example, the National Association of Colleges Employers Survey found that in both 2015 and 2016, the highest essential need rating employers gave to skill sets was given, of course, to critical thinking and problem solving. It’s the number one essential need. Yep, so at a future points list, more and more research that’s showing the same strong desire for critical thinking from employers and from educators alike. But that’s one example. Now, if we contrast that against where we are standing as a nation with respect to critical thinking outcomes, the Department of Education did a big study where they’re looking at problem solving in a technology rich environment, and we’re referencing specifically the OECD twenty sixteen. Skills Matter further results from the survey of adult skills and what that study shows is that among millennials out of 19 countries, the U.S. ranks. 18th, in critical thinking outcomes or problem solving in a technology and rich environment among millennials.

Dave Carillo: That’s not a great place to be, actually.

Steve Pearlman: And we did beat

Dave Carillo: Poland, we beat Poland. So just to frame this, even though critical thinking and problem solving is one of the most or the most sought after skill by employers today, the United States students are like falling way behind in terms of actually being able to prove that they have that skill or have that skill at all.

Steve Pearlman: And we know, by the way, look, there’s debate within the critical thinking world as to whether critical thinking and problem solving or synonymous. Sure. And we’re not going to get into that debate here today. Let’s say, for argument’s sake, there’s enough overlap between those two concepts, at the very least, to make that reference relevant to the conversation.

Dave Carillo: So what we’re seeing, though, is at least one major gap in what students need to succeed outside of college and what they’re actually leaving college with. And you know, again, Steve said something earlier. This is not going to be an argument that places the most value on simply getting a job, but getting a job is kind of important, too. So that’s one way we can start to think about why we need to continue to talk about critical thinking, continue the conversation, raise these outcomes, strengthen student critical thinking across the board.

Steve Pearlman: So if you’re on this podcast, you’re probably listening because you are sympathetic in a certain sense to the problem of critical thinking outcomes. You’re quite potentially an educator who is not fully satisfied with the critical thinking outcomes you’re seeing in your students or want to be able to engage them more deeply on critical thinking levels. This, of course, brings us to the obvious and important question what are the reasons you believe that critical thinking outcomes are so low?

Dave Carillo: And if you’re baffled, what have you heard in terms of conversation? I’m sure you’ve been in a conversation with someone who seems to think that they have something to add about this. And so what would they say or what have you heard people saying as to the reason why students are unable to think critically the way they ought to

Steve Pearlman: Be talking to so many educators about this, as well as people in industry and so forth? What we hear are a remarkable number of reasons from popular conceptions. We hear things like, well, it’s because of those darn interwebs and how students are overwhelmed with information, and they’re not thinking about any of it anymore.

Dave Carillo: Another popular one is the standardized tests, which we are not fans of. But you’ll hear often that standardized testing is limiting student’s ability to develop and strengthen as critical thinkers.

Steve Pearlman: We actually have heard arguments that students these days just are not as smart as students of previous generations. Yes, exactly. They’re getting dumber argument, and we certainly know that in a lot of classes there are pushes on content and so forth that have to be achieved. And we want to be really clear here because there’s no question that standardized testing is not contributing to the solution. Standardized testing is in no way going to improve our students critical thinking abilities. But what’s essential here that we want to point out is that the mere removal of standardized testing does not necessarily also increase critical thinking outcomes. That standardized testing, though problematic, is not the problem because it’s elimination doesn’t foster critical thinking growth.

Dave Carillo: No. Likewise, overreliance on technology or inability to read the internet or use the internet in the right way. These things are going to crop up in our future discussions. But just like simply removing MEST as testing isn’t going to raise critical thinking outcomes, disallowing students from using Wikipedia is not going to do that either.

Steve Pearlman: Nor are critical thinking outcomes going to be raised simply if we get every kid a tablet. So the absence of technology or the presence of technology is not necessarily directly affecting the critical thinking issue.

Dave Carillo: No, it seems like the kind of thing we’re talking about here. Are these more sort of popularized conceptions of what’s gone wrong with students today? Yes, but we those damn kids, those damn kids. But that’s an argument that’s been made for the last ten thousand years. Kids are always going to be kids. What we’re here to talk about, though, is that we we’ve been doing research on this for quite some time, and we think we’ve started to uncover some of the core reasons that critical thinking outcomes are not as high as they could be.

Steve Pearlman: I like how you point that out, though, that it’s this is an age old argument. Can you envision Neanderthal grandmother being frustrated with, you know, Neanderthal child’s use of the rock the rock?

Dave Carillo: Well, there was that. That’s famous. No. I think I’m going to have to look this up, I’ll bring it in because I seem to remember reading a passage from a Roman writer talking about how teenagers had no respect for adults during the Roman Empire times. And so, yeah, this is this argument that persists, and we’re not here to talk about the internet being bad or, you know, standardized testing being evil, although that might come up. What we’re going to talk about actually arises from the research we’ve done in the faculty development that we’ve done over a lot of years. So we’re not just pulling five new things out of the air to be different. No, that we’re not just consulting a Ouija board on this.

Steve Pearlman: Not just consulting,

Dave Carillo: Not just consulting a Ouija board on

Steve Pearlman: This. So let’s try to conceptualize this in a certain way now, because it’s easy to. I think both of us could continue railing against prevailing norms for a long time. Well, we want you to consider, is this an ecological or biological ecosystem? Has certain forces that enable it to function and enable an organism within that ecosystem to survive, if not thrive. We know there has to be sunlight, there has to be water, there has to be food, there’s a food chain and so forth. All of these things enable any organism within the ecosystem to be able to grow and survive. What we postulate is that education needs to serve equally as an ecosystem, but instead of a biological ecosystem, it has to be a critical thinking ecosystem, and it has to be an intellectual ecosystem, meaning that all aspects of that intellectual ecosystem, all aspects of that educational institution or all aspects, at the very least of that individual classroom have to align to support one another, such that the individual intellectual organism can survive. Imagine that there is this little brain emerging into your classroom or into your campus, and we use a a brain with lizard

Dave Carillo: Brain with legs

Steve Pearlman: As an image

Dave Carillo: On our PowerPoint.

Steve Pearlman: And what we want to ask you to think about is if there is this intellectual organism emerging into your institution or into your classroom, has it become a stronger brain? Has that little brain become a bigger brain, right?

Dave Carillo: So the same way that if you want to keep your cat alive in this biological ecosystem and you remove the sun, the cow is going to die because the grass is going to die and so on and so forth. What we’re going to talk about here are elements that, if not align, if not in place, our little brain with legs will not be able to thrive as an intellectual organism.

Steve Pearlman: So if those of you listening were going to construct an educational ecosystem that had a number of different parts to it that all had to work together, what are the parts of that ecosystem? What does the intellectual organism need in order to thrive? So, for example, rich texts, rich reading materials as being part of a classroom as being essential to the development of critical thinking. We often hear things like access to technology as being critical.

Dave Carillo: Sure, some faculty folk would say just the opportunity to think critically. If you’re teaching a class of 300 kids, if there is a major content push, then there might not necessarily be the time where you might not conceive of there being enough time to even work with the idea of critical thinking in a class. So just the opportunity itself, right?

Steve Pearlman: A smaller class sizes. Yeah, absolutely. The ability to have intellectually rich discussion with students or the ability to engage students in writing assignments. These are all things that are often associated with what would comprise a critical thinking ecosystem, and lots of those things can be very valuable to education and valuable to a thinking rich environment overall. But they’re not the essential forces of a critical thinking ecosystem as we’re identifying it. So, right, so let’s not tease everyone any longer. Let’s go into what that looks like.

Dave Carillo: Ok, read the list.

Steve Pearlman: We want to first emphasize something, which is that as we initially give you the list, it might seem somewhat obvious that these things must be present. But what we’re going to show over the course of the podcast is how absent these things are. In fact, from education on a whole. So bear with us, it sounds like we’re listing just a bunch of obvious things that anyone could think about. Yeah. So in order for a critical thinking ecosystem to manifest, we’re identifying five key elements that must all be aligned for the intellectual organism to grow. If any one of these things are out of balance, it can jeopardize the entire existence of the ecosystem. First, it’s going to sound obvious we need research driven practices in the classroom, not practices based on what may have worked on personal. Experience or what our suppositions are around teaching and critical thinking, that’s first second. The assessments that happen in class must be focused on critical thinking rather than focused on content acquisition or other skill set. Third, pedagogy and assignment design must cultivate critical thinking in itself. There are pedagogies that are associated with this, and there are pedagogies that are not strongly associated with the growth in critical thinking. Fourth, we need to offer students a process through which to think critically. We need to identify for them what they can actually go and do in order to think through any given idea. And finally, we need an interdisciplinary approach to this critical thinking ecosystem, such that the conception of critical thinking doesn’t change radically as they move from one discipline to the next, but rather that at least in some fashion, we are reinforcing the same skill sets for the

Dave Carillo: Student, right? We want students to be able to develop their ability to think critically. We don’t necessarily want to have them to have to start over class by class by class.

Steve Pearlman: Now we know that there’s a lot of discourse around the question of whether critical thinking is something that’s called domain specific, which means disciplinary focused or whether it’s a more generic skill. So in other words, it’s something that is different entirely in a literature class than it is in a class in psychology. We’re not going to get into that debate right now. Suffice it to say we will in the future, but we do believe that there are some generalizable skills, and

Dave Carillo: That’s good evidence to suggest that. That’s right. Critical thinking is a general skill that can be developed and applied to any discipline roughly in the same way.

Steve Pearlman: That’s right. We will deal with that debate later on in another

Dave Carillo: Podcast and or ask questions, and we’ll try to get to your questions about that in the next

Steve Pearlman: Podcast. So let’s go back to the first issue. Here are our practices in academia generally driven by research, and the answer to that is no, generally speaking, practices in academia, or at least certainly in higher education. More specifically, and we can’t speak to everything at once today or not. So I’m going to read from learning our lesson a review of quality teaching in higher education. And this was a study done again, as initiated by the Department of Education. What I want to be clear about going into this is its use of empirical in this particular study is speaking generally to what is observed rather than what is evidence driven. So there is a connotation around empirical that might suggest that it’s coming out of research. And in this case, they’re using it more in terms of what’s observable. And the study comes to some conclusion here quote teaching matters in higher education institutions although quality teaching encompasses definitions and concepts that are highly varied and in constant flux. There is a growing number of initiatives aimed at improving the quality of teaching. However, the study goes on to say the vast majority of initiatives supporting teaching quality are empirical and address the institution’s needs at a given point in time. Initiatives inspired by academic literature are rare. I’ll emphasize that point again. Initiatives inspired by academic literature are rare. In other words, what we generally see happening at our institutions is that faculty will or the institution as a whole will identify a particular need in the students for them to do more of this or do more of that. And they will move based on what they observe as that necessity and what they observe as being useful remedy to that, and less so based on collectively speaking, can speak to your individual classroom. But collectively speaking, they move less on what the research actually supports in terms of an effective means of contending with whatever the observable problem might be.

Dave Carillo: Yeah, and this is always been a tough issue for us to broach with faculty because it seems counterintuitive to plan a class or make a departmental decision based on, you know, research done at another school right with other students when we can see our students right here. But what we’re seeing, though, is a real lack of utilizing the research that’s out there to make the kinds of decisions that can incrementally improve a classroom and institution teaching style pedagogy. And then the other issue, too, and we see a lot of this. If something’s worked in a classroom, why change it? Right, right. And to some extent, we get that, except that what we’re seeing from the outcomes nationally is that the critical thinking outcomes are not as high as we would want. So whether it’s working here and there or whether we think something is working, it might not be actually working, actually working as good as we would want to.

Steve Pearlman: There’s that difference between what we’re perceiving as effective what. Our students are perceiving as effective what the institution is perceiving and effective and what research is actually supporting is being effective practice. And what this study affirms, and it’s only one of many because we’re only highlighting this is

Dave Carillo: A very high level first podcast of a lot of the issues that we’ve looked at in depth.

Steve Pearlman: But what that study is supporting is that on the whole, academic institutions are not driving their practice from research on pedagogy, and it’s up to every individual to think about the extent to which that applies to their classroom or their institution. But that’s the nationwide research that’s coming out. The next thing we’re going to look at is assessment of everything that’s going on in the classroom. The single element that most drives what students do with respect to how they approach their learning is in fact assessment. So I’d like to read here from Strathaven, Doshi and Jansen’s students perceptions about evaluation and assessment in higher education, a review which appeared in Assessment and Evaluation and Higher Education in 2005. Quote the reviewed studies, evidence that students perception about assessment and their approaches to learning are strongly related to perceived characteristics of assessment seem to have a considerable impact on students approaches and vice versa. These influences can be both positive and or negative, especially assessment procedures that are perceived to be inappropriate. Ones tend to encourage surface approaches to learning. So to sum that up, what this study and their multiple studies are showing is that how students perceive they’re going to be assessed as a driving force in what they go to do in the class, how they approach their learning experience in the classroom or how they approach their study methods. So with that in mind, the question becomes this if assessment is in fact arguably the largest driving force for what students go and do in the process of their learning are most educators using an assessment where primacy is given to assessing critical thinking?

Dave Carillo: Sure. And I’m going to follow that up with a very common question. I think that a lot of our listeners have heard, which is just tell me what you want me to say or just tell me the right answer. And that does point to the idea that students are focused on the grade. We would want that assessment to speak to critical thinking rather than just the right answer. We would want that assessment also to be transparent,

Steve Pearlman: And we’ll talk about that more when we talk about assessment in a future episode. But look, what this is saying is really clear. If students don’t find that assessment to be an authentic or a valuable assessment, they’re just going to focus on getting the grade. Yep. Most educators, unfortunately, are not using rubrics or using assessment methods that explicitly and primarily focus on critical thinking. Rather, they’re focusing on content absorption or other factors, or perhaps even implicitly, in some indirect ways, valuing critical thinking, but not directly and not giving primacy to it. As such, students are not curbing their learning methods so that they are developing those critical thinking skills or trying to do as much critical thinking as they could.

Dave Carillo: We don’t want to make the grade dependent on how much a student can memorize and then turn around and ask why there haven’t been thinking critically about the in the material. All they need to do to get a good grade is to memorize a bunch of facts. Then that’s really all they’re going to go out and

Steve Pearlman: Do, and that’s the right thing for them to do in a lot of respects, right?

Dave Carillo: If that’s the other thing, that’s

Steve Pearlman: The problem, right? There’s a lot of controversy now, certainly in education around how much importance is being put on grades and so forth. And what we’d like to make very clear here is that we are equally concerned about the importance being put on grades as having a negative impact on students, but only when those grades are not really authentically measuring critical thinking. If the students are really being assessed on critical thinking and critical thinking is seen by the students and it typically is as inherently valuable, then grading an assessment doesn’t have to be the boogeyman of education. Grading an assessment can be a positive force in education students can appreciate, and we’ll talk more about that in an episode to follow.

Dave Carillo: Definitely.

Steve Pearlman: Ok, so let’s put this again in perspective. What we’re saying is is the educational institution, a critical thinking ecosystem. And what we’re finding first is that on the whole, the practices are not driven by research around what is supporting critical thinking. Second, what we’re seeing is that with assessment being the strongest driving force for what students actually go and do in the process of their learning, we’re not seeing assessments that are focused primarily on critical thinking and assessing critical thinking and valuing critical thinking. Other things in the classroom are generally valued instead. Third, let’s look at pedagogy now. Does pedagogy line up as part of this ecosystem? Well, a recent S’engage study, which you can find at the S’engage blog, asked faculty the extent to which they are teaching critical thinking and the vast, vast majority of faculty. In the study, roughly around 90 percent said they absolutely teach critical thinking and value critical thinking highly as the most important thing that they’re doing in their education. So then they were said, Well, what are the things that you’re doing in order to teach critical thinking? And the top four answers were group discussion, case studies, reading questions and online discussions. Now, I think many of us would associate some of those things with really cultivating critical thinking in our classroom.

Dave Carillo: Well, they seem like very common elements to virtually any classroom group discussion. That’s huge.

Steve Pearlman: A good reading question, right? The problem is that all of those methods are what are referred to in the critical thinking field as immersion methods. Ok. And an immersion method of teaching critical thinking is functioning on the premise that by involving students in a activity where thinking could occur, that thinking therefore not only must be occurring, but students must also in fact be learning to become better at it. And the research unfortunately does not support that. Immersion methods of teaching critical thinking actually create lasting critical thinking outcomes, especially in the sense of being a generalizable skill that moves from discipline to discipline that stays with the students as they move forward. There’s a lot of different research on this, and there is some discrepancy as to the value of immersion methods. At least within an individual classroom, sometimes it seemed to be somewhat valuable, sometimes it’s not, it’s never shown to really be persistent outside the classroom. In fact, a very recent article, which is a part of a long list of articles on this subject matter, just published in the International Journal of Science and Math Education by Tirana, Derkach and Ellen, entitled Designing Learning Environments for Critical Thinking, examining effective instructional approaches actually looked at this question, and they didn’t compare it to direct instruction, which is what we’ll talk about in a second.

Steve Pearlman: But they did look at immersion methods and infusion methods, and diffusion methods are kind of somewhat talking about what a disciplinary specific concept of critical thinking would be within a discipline. Whereas immersion is just doing an activity generally that we hope involves critical thinking, and what the study found again was that although with some very careful planning that they put forward in order to construct and design a very specific learning environment, the immersion methods and infusion methods did produce some critical thinking gains in students within that specific environment. It failed to produce any critical thinking gains as the students moved into a generalizable skill. So in other words, students became better at doing something within that class that was considered to be critical thinking within the scope of that class. But as they were asked to do it outside of that immediate classroom, they showed no improvement whatsoever.

Dave Carillo: Interesting.

Steve Pearlman: What’s interesting about this is that in fact, immersion methods that are listed in that s’engage gauge study where we’re talking about reading questions and so on are actually far broader and far less developed than what the people did in this study, which is why there’s such a range of results around immersion and infusion methods so that very often immersion methods are shown to do nothing at all. Why? Because merely involving students in a group discussion broadly actually is shown to do nothing, necessarily with respect to improving their ability to think critically, either within that environment or moving out of that environment. With a lot of careful planning, immersion and diffusion might be shown to be able to cultivate something within that individual environment, but it doesn’t seem to be transferable or portable outside that environment. So what are we seeing then? The vast majority of techniques that faculty say that they are using in order to teach critical thinking are not only immersion methods, but there’s such broad examples within those categories of immersion methods that they’re probably having virtually no effect on the positive outcome of critical thinking at all, which again relates back to why critical thinking outcomes in the United States are so low because faculty here who are saying that they’re using group discussion and eighty two percent of these faculty said they use group discussion to foster critical thinking. Those faculty members undoubtedly believe that that group discussion is improving critical thinking of their students when, at least according to the research on the whole, we can say there’s probably exceptions, but on the whole, we can say it’s not. I certainly was guilty of that same kind of thinking back in my teaching career.

Dave Carillo: Well, absolutely. And I’ve seen I think we’ve all seen and this is, you know, something I guess it runs contrary to our argument that, you know, we don’t necessarily want to rely specifically on our own experience. But I think in this case, we can at least reflect on how many times group discussion has developed into our just asking questions until someone decides to answer that question or some sort of version of the Socratic method, where maybe there’s a set of right answers we’re looking for and we’re going to just continue to draw on the students until they get to that point or if we put students into a group to discuss something. How many times has that group started discussing that subject, that material and then broke it down into what’s going on on campus later on that night?

Steve Pearlman: And so that’s a great point. Even if we have a great discussion with the students in our class and we leave that class feeling as though we did something great and perhaps very well did within the scope of that individual discussion. The problem is that we’re not really giving students a sense, nevertheless, of the process that goes into thinking critically about something. Even if we led them to some thoughtful ideas, we didn’t lead them into the process, right?

Dave Carillo: Even that study took incredible pains to design these methods to see what could come out of them. And that’s not necessarily something that we can always do. It’s not necessarily something that is always on our mind in terms of something like a group discussion or a case study or we write and that kind of thing.

Steve Pearlman: So so this goes into something called like the deep structure of learning. I can follow a recipe for how to bake a cake, which might be like being led through a great group discussion, and I can come out with a great cake at the end of that. I haven’t learned how to bake a different cake. I haven’t learned the principles of the ratios that I need to have between baking powder and flour and sugar and so forth. And those are the things that are the deep structure that I need to. Understand not just how to follow a recipe in order to get to the cake. The answer to that issue is, in fact what it’s called direct instruction of critical thinking, which is teaching critical thinking as a subject of itself so that we’re walking students through a critical thinking process, right?

Dave Carillo: It’s like not just the discussion, it’s stopping to reflect on what’s happening in the discussion, the assumptions made in the discussion, developing that sort of metacognitive framework for looking at the discussion and what’s coming out of it and what’s going into it and so on and so forth, which is actually where we’re headed next, right?

Steve Pearlman: Well, first, let’s talk for a second about assignments as well. I don’t know. Absolutely. So part of pedagogy is assignments and a great book. Assignments across the curriculum makes it pretty clear case that most assignments writing assignments we’re talking about now are not explicitly asking for higher order skills. The vast majority, about two thirds, are asking almost exclusively for summary and regurgitation of material. And sometimes that’s certainly valuable. Absolutely. But if we want to look at whether or not we’re moving students towards stronger critical thinking skills, we find that both pedagogy that we’re using and the assignment designs that we’re using on the whole aren’t forwarding students towards the development of those skills.

Dave Carillo: Absolutely. We can start to see a little bit of the way where we want to talk about the relationship between these elements in the critical thinking ecosystem and how they have to be aligned if we’re going to test students specifically on what they’ve been able to remember, right? And that’s what the assignment is asking for them to do. Just repeat what I said in this lecture or take five sources and summarize these sources, if that’s what we’re going to be assessing on. If that’s what the assignment is asking for, then we’re not going to get that kind of critical thinking because it’s just not there for the students to do.

Steve Pearlman: So when you’re trying to build this critical thinking ecosystem, we know the importance of this critical thinking ecosystem, but it’s not research based. We don’t see assessments that are emphasizing critical thinking. We don’t see pedagogy in assignments that are calling for critical thinking. And then that brings us back to that. The fourth, which is really the process or direct instances, are we giving students a literal process to move through in order to contend with problems or to think critically about a source in order to think critically during a conversation? And again, that goes back to our pedagogy and it’s overlapping. But the answer again is no. On the whole, students are not receiving direct instruction in actually how to go about thinking critically. One of the reasons we know that faculty resist that is a it’s hard to identify what the critical thinking process or a valuable critical thinking process would be for students. B, We know that faculty feel as though we have to focus on content in our classes, so we don’t feel as though there’s necessarily the time to do that. And see, even if we do think that we might have a conception of critical thinking that we can teach to students or transfer to students for students to use as a process. Are we limiting them in that conception?

Dave Carillo: We, you know, in our research and our experience working with faculty on critical thinking and raising critical thinking outcomes, we see a lot of things like analyze this text or evaluate this evidence. But those words are extremely complicated words, and those moves are extremely complicated moves. But we never see any explication of what goes into an analysis. What do you mean by evaluate? And so if there isn’t that kind of direct process, the students are left sort of high and dry as to what to do to analyze.

Steve Pearlman: And this goes right back to what we were saying about assessment and what goes right back to what we’re saying about pedagogy and assignment design that even if an assignment calls upon students to say, analyze this text, the argument could be made that the assignment is in some way calling for critical thinking. But if the assessment isn’t defining what analysis is and students don’t have a process through which to go through analysis, then they’re not going to do it because they simply don’t know how to do it. And that’s where this ecosystem model comes into play. Because all of these factors the assessment, the assignment, the pedagogy, the process for students, they all have to align. And if they don’t align, then the students aren’t going to be able to perform the way we’re hoping.

Dave Carillo: It’s really why the critical thinking and initiative exists is to bring these kind of concepts to folks and help them to employ them in their classroom.

Steve Pearlman: Well, that really brings us into our fifth the fifth element of the ecosystem, which is that it’s used in multiple classes and multiple levels in multiple disciplines.

Dave Carillo: Exactly. But even before that. And this is something I think we could dedicate a podcast to. And you’ve said this before, but there is no content without thinking about the content. You know, a student can be inundated with content. The student can be lectured to perfectly and a student can be given every name, place and date from now until the end of time. But if they’re not engaging the content critically, it’s not going to matter to them in any way, and it’s eventually going to leave their brain. The content is just not secondary, but it’s suffered. Greatly, if there is in that sort of critical thinking ecosystem,

Steve Pearlman: Well, that’s the thing, right? So we can either view critical thinking as a burden on our delivery of content because we have to deliver this body of material to the students in this class. And I don’t have time to teach them how to think critically about it. Or we can look at critical thinking as the doorway into being able to engage that content in the first place more deeply to increase persistence of that learning into other classes. And that brings us again back to that idea that the fifth element of the ecosystem is that it has to be interdisciplinary. It has to work in multiple levels. And so at USDA, we’re fortunate now where students can enter a 200 level class already having been exposed to the system in a 100 level class. So the faculty member in the 200 level class is less burdened by having to reconstruct students in the process initially or initialized students into their process and can hit the road running by assuming that the students know what they’re going to mean when they say that they want them to write this kind of paper.

Dave Carillo: Right. And when we talk to faculty about this, you know, we say it needs to start in the classroom, obviously. Would it be the burden shouldn’t just be on one individual to teach critical thinking to these kids and have them get it and have them move on to whatever. And in fact, that’s something that that strengthen the price of it does. It takes a village. The more classes are doing it, the stronger the biology department, the more classes that are doing it, the stronger the school of pharmacy.

Steve Pearlman: That’s exactly it. So let’s return to our ecosystem model. Sure. And let’s reframe the question back to our listeners. Are all the practices in your classroom and all the practices in your institution really framed around the research of what drives critical thinking outcomes? Are your assessments giving primacy to critical thinking as the most important thing that you’re assessing the students on? And is that exceptionally well defined? So it’s not enough to say, for example, that this is good analysis or very good analysis, but are you really explaining and delineating for students what those aspects of critical thinking really are? Third, is that assessment being supported by pedagogies that actually support the growth of critical thinking in the classroom. Group discussion, for example, might be rich and might help students to understand given body of material and might raise some really interesting ideas. But that doesn’t necessarily mean that critical thinking in itself is being cultivated within that structure. Fourth, are you supporting that pedagogy with a direct process that students can use to actually achieve the critical thinking outcomes that you want? Is that process for students the same way they’re going to be assessed? Is that process for students the same thing you’re asking them to do in your assignments is that process for students the same way that you’re teaching every class so that you’re reinforcing the same kind of concept over and over again are all these parts of the ecosystem aligning.

Steve Pearlman: And if all those things are even present in your classroom, are they present in the same fashion or at least a strongly similar fashion as they move into the next classroom? If every teacher at the university is teaching a competing conception or holding a competing conception of critical thinking, it’s difficult for students to ever master one conception of critical thinking. And we want to be careful not to limit students to only one way to think about thinking, but at the same time, if they’re going from class to class and always just meeting with different expectations and those expectations are typically undefined, then they’re never going to really cultivate any one skill set in critical thinking at all. And that’s kind of what we’re seeing now in terms of the nationwide outcomes. I do think it’s time that we shift to our news of the week segment. Absolutely. But first, this word from our sponsor.

Voiceover: This episode of the Critical Thinking Initiative is brought to you by the Critical Thinking Initiative. Org Visit the Critical Thinking Initiative. Org For an ever expanding resource of books, syllabi, videos and other resources on critical thinking, the critical thinking initiative is always seeking out other people in education who want to work towards raising critical thinking outcomes locally and nationally.

Dave Carillo: All right, so who wants to go first? Steve, you you’ve got got your papers in hand so you can go first,

Steve Pearlman: If that’s the way we’re going to determine this moving forward. I’m even more conscious about whoever

Dave Carillo: Whoever picks up their thing first gets to go first.

Steve Pearlman: That’s what it is. I’m going to talk about a recent article in Twenty Seventeen from Science and Education, which is by Ann Collins, McLaughlin and Alicia Abbott. Mcgill, entitled Explicitly Teaching Critical Thinking Skills in History Course. What they did in very broad strokes was they controlled for some students who took a research methodology course and compared them against students in history course, where they taught some explicit critical thinking skills to the students are focused explicitly on critical thinking and history. And they wanted to see does the generic skill of looking at research methods improve students relative to explicit instruction in history? One of the most striking motivations for wanting to do it was the perception that critical thinking can only really be taught explicitly in the stems because science is a critical thinking method. And there’s this perception they were experiencing that critical thinking couldn’t be cultivated as well. And a humanities environment, we’ve

Dave Carillo: Seen that before. We’ve seen that argument. We’ve seen

Steve Pearlman: That argument. And I think they were experiencing that argument. So I’m excited to see them go after that perception and challenge that perception outright. And they did so and they did so fairly successfully in the sense that they showed students coming out of the research methodology course did not necessarily translate those skills over to specific other contexts in terms of critical thinking and that they were able to meet with some success in terms of cultivating critical thinking in their students. In the history course, where I’m a little softer on their achievement is that their approach to critical thinking was looking at inherently flawed historical conceptions, such as the idea that the pyramids were built by aliens. Ok, OK, so wait, what did you not know that they were? They weren’t. Well, we can’t say for sure.

Dave Carillo: Ok. All right. All right.

Steve Pearlman: Good enough. So they they wanted students to be able to scrutinize these popular misconceptions, sort of of history and be able to examine text and determine what was flawed about some of these arguments being made about at least why the pyramids must be have been made by aliens. I’m not saying they’re not Dave. I’m just saying some of the arguments. They hear some of the arguments being made that they

Dave Carillo: Are not a worldview,

Steve Pearlman: The most intellectually cohesive arguments being made in the intellectual.

Dave Carillo: I see. Ok.

Steve Pearlman: So that’s a valuable skill for students to have is to be able to separate false and and fallacious logic and fallacious argument from true evidence based work. And I applaud them for being able to go after that and for cultivating that skill, and students got to start somewhere, right? Sure. So what I feel is most important here comes in their conclusion at the very end quote, the current study has shown that critical thinking skills can be taught via a non-science course, resulting in a reduction in pseudoscience and paranormal beliefs. End quote Good. Glad to see that, but one of the biggest takeaways they end with here is this quote Cross-disciplinary and collaborative scholarship on teaching and learning of critical thinking skills could also be particularly valuable and quote, in other words, and what they lead up to in saying that is that if students are not given a similar conception, nice of critical thinking across the disciplines, if this is not reinforced, merely teaching them research methods is not necessarily going to cultivate in them strong critical thinking. And they don’t emphasize that point, as I’m emphasizing that point. And so there’s some part of this that’s me resonating with something they said there. I don’t want to overstate their case, nevertheless, that they they arrive at that point after this study, to me is particularly important. I think what they’re alluding to here, at the very least, or they’re leading me to think about is that they’ve had this success with students with respect to a cultivation of critical thinking. And they’re recognizing that the value of, if not the need for that conception, the critical thing that skill set in the students to be reinforced class to class, subject to subject discipline, to discipline as they move forward.

Dave Carillo: I like that you’re bringing that point up. We’ve seen that before, too. In other articles, the sort of it’s not a throwaway. So or it’s not a throwaway sort of statement of implication, but it’s this implication that we’ve run into it is it’s valuable to see it in other studies, at least that they’re recognizing it, right?

Steve Pearlman: Well, in fact, what’s interesting about it too, and I like the point that you’re bringing up here is how often we’ve seen it. We’ve typically seen it as. And by the way,

Dave Carillo: By the way, this would

Steve Pearlman: Be great. This would be great if it was reinforced in multiple classes and then we don’t see many studies. Where that’s actually happening, because it’s not happening in a lot of institutions, and so we are in a position to be able to start to speak to what can happen when we reinforce the similar conception across multiple classes. But pulling us out of the equation just that that stated as something that’s needed repeatedly and b that it’s stated as something that’s needed and not tested speaks to where the field of education needs to go. If we’re going to develop critical thinking skills in students coming out, I agree.

Dave Carillo: I couldn’t agree. I couldn’t agree more. I have a question for you. When was the last time you were? You were at Gwyneth Paltrow’s Goop website?

Steve Pearlman: I’m sort of to

Dave Carillo: You, I know they’re often I’m

Steve Pearlman: Actually sort of proud to say that I’ve never been do Gwyneth.

Dave Carillo: This is a follow up. Do you know what the Goop website is about? Do you know what Goop is? Do you know what Goop is?

Steve Pearlman: I don’t. I’ve heard things.

Dave Carillo: What about is Gwyneth Paltrow? What is Goop? Well, we’re not here. This is not an ad hominem attack. No.

Steve Pearlman: Some of the things she said that have not necessarily panned out to be scientifically scientific.

Dave Carillo: If our listeners are aware the Goop website is Goop is Gwyneth Paltrow’s lifestyle company, and

Steve Pearlman: She named a lifestyle company Goop GOP.

Dave Carillo: I don’t know the history, but it might have started with some sort of like skin creams. Maybe your hair stuff was it was all natural, but is this all natural? The whole thing is all natural and organic. Well, what

Steve Pearlman: Is it like, Gwyneth? I don’t she, Gwyneth.

Dave Carillo: I don’t know of her. No, I have not. It’s not an acronym as far as I’m OK. It’s not an acronym at any rate. This is from And I’ll be honest, I’m not. I’m not a regular reader of The title of the article is it’s Gwyneth Paltrow pseudoscience winning. Ok, so let me just sort of set up this this argument here, because there are a couple of things that I want to get to in this that that I thought were really interesting. But essentially the subheading of this right. The subheading of this article is Goop has been called out or the title line or whatever. The Goop has been called out for bullshit over and over. But the brand seems to be stronger than ever. So essentially what’s what’s happening here is that the Goop website, or the lifestyle company apparently advertises for a lot of health and health related products that seem to have no actual scientific basis or validity. And so the ones that they explain, the ones they include are things like sex, dust and body stickers that promote healing and things like that. And so the author of this article, who is Julia Bayless, she sort of goes into this argument that there’s multiple doctors and scientists have called the Goop company out for promoting products that just have no scientific backing.

Dave Carillo: Then they have a picture. They have a picture of this jade egg here that retails for sixty six dollars and they say this yoni eggs. Once the strictly guarded secret of Chinese concubines and royalty in antiquity, harness the power of energy work, crystal healing and key gold like physical practice. The ongoing conversation is that Goop advertises these products that are clearly not in any way scientifically backed. Scientists and doctors debunk these products, and the company just seems to be growing stronger. This raises a lot of issues in terms of ideology and what I wanted to get to. Here are a couple of things quote still, wouldn’t the negative press surrounding health claims have made some dent in their business? This individual that she’s interviewing, Avery, doesn’t think so. The old adage no news is bad. News comes to mind here. And so essentially what’s happening is that although these products are being debunked, the brand is growing even stronger. The author proposed posed this question to Larry Light., author of Six Rules of Brand Revitalization and the chief executive of the brand consulting company Architecture. And he says you can’t attack a belief with facts, which is what I want to kind of get to right.

Dave Carillo: He agreed that Goop debunking would only galvanize its fans and thought that Paltrow’s new summits in magazine would further expand the Goop cult and deepens in its members beliefs. So what we have here are a bunch of experts saying this jade egg is not going to do anything for you and a bunch of people saying We’re going to go ahead and buy the jade egg anyhow. The article takes a turn when the author starts to ask doctors and health researchers what their advice is for best practices in fighting this kind of quote fake news and we see fake news everywhere now, right? There’s a lot of fear. It’s not just political, it’s Goop related fake news to someone introduces a jade egg and someone buys, said Jade Egg, and claims that it helped them. And then a doctor comes around and says, there’s literally nothing to this. Jada, again, the person goes out and gets a second jade egg. Some of these health professionals came around to a belief I’m now wholeheartedly convinced of, says the author of this article. The best way to stop bogus health claims from taking off is to teach people how to think critically about the information they receive from a very early age, which I find is very good advice.

Dave Carillo: I brought this to the table one because again, yes, I agree. I think you would agree. Let’s teach these students these individuals to think critically about the information from a very early age. The challenge has always been how are we going to define it, how we’re going to teach it, how are we going to assess it? One of the things that I want to further. Research, but I’m going to talk about here because I’m just really interested, is that researchers from the article goes on to explain that researchers from Europe and Africa recently worked to develop curricula. It’s a cartoon filled textbook, lesson plans etc and critical thinking skills aimed at schoolchildren in twenty sixteen. They tested this on 15000 schoolchildren from Uganda’s central region. I don’t know enough about Uganda, so there might be some context there. Maybe there are a lot of misinformed students. I’m not sure. The results of the trial, though, were published in The Lancet in May. The very well-regarded and long standing medical journal from England that showed a remarkable rate of success. Kids who were taught basic concepts of how to think critically about health claims massively outperformed children in a control group. Huh. Right, right. So there you go.

Steve Pearlman: Let me jump back because something you’ve said is very interesting here and I want your take on it. And I don’t think there is. I don’t think this article offers a clear example. I just want to explore it for a moment. The central premise that we’re seeing in the article really is that information will not counter other information that even though the Goop information is unfounded, that the presence of counter information to information, even if it’s more logical or from stronger authority or better evidence isn’t enough, it’s the individual’s capacity to reason that is the difference maker. Is that what this is getting? I would say

Dave Carillo: That this is exactly what it’s getting at time and time again. Goop apparently introduces a product for a premium price that claims to have some sort of benefits for an individual’s physical well-being.

Steve Pearlman: Well, I just say you don’t know that it’s a premium price. I mean that Goop Egg could

Dave Carillo: Be jade egg.

Steve Pearlman: Maybe that’s cheap for a Chinese concubine Peruvian jade egg. I don’t know what those should go for.

Dave Carillo: That’s a fair

Steve Pearlman: Point, right? That could be. It could be a deal that could be a deal.

Dave Carillo: It could be a deal. We don’t we don’t make our way to China and hang out with concubines from antiquity enough to really get a sense of what the price ought to be.

Steve Pearlman: Maybe we should do that. More is

Dave Carillo: An experimental, potentially

Steve Pearlman: Acquired antiquity thing

Dave Carillo: Choir time machine. Yeah. Head to China. At any rate, all right. You’re right. The overall thrust of the first part of this article is that yes, regardless of who’s debunking these in regardless of of the place of authority that they come from scientists and health experts and doctors, it has not in any way diminished the power of the brand to sell these kinds of products. And often, the go to for dealing with this is this more foundational ability of the individual to think critically about the relative validity of the information? What we’re seeing here implicitly is just how powerful the myriad cognitive biases housed in the human brain are when

Steve Pearlman: Power of belief persistence.

Dave Carillo: Right, right. I feel better. I have this jade egg. There must be some relationship, right? God forbid that that relationship is shattered by other evidence and not. But that’s that’s a very, you know, that’s that’s a very common bias and that’s something that we have to all work at and there are a lot of them.

Steve Pearlman: So none of us are free of that. This is probably a fairly demonstrative example of this problem, but it’s something that we are all contending with.

Dave Carillo: Exactly to a certain degree, we’re all contending with. And maybe one day there’ll be a jade egg for that. I’m not sure.

Voiceover: Thank you for listening to the Critical Thinking Initiative podcast. Got questions for Steve and Dave? Just send them to info at the Critical Thinking Initiative dot org. Follow us on Facebook and Twitter and find us at W W Dot The Critical Thinking Initiative. Org.


Submit a Comment