Assessing Learning - Example Scenarios
Imagine a classroom with a student, a teacher, and an impartial objective observer. For this you can even imagine the observer with the usual white lab coat and clipboard taking notes. For 21st century classooms we'll allow a tablet, but the white coat is still helpful. Let's take a look at different scenarios and student projects and try out these tests to see which parts qualify as recall, comprehension, application or synthesis.
Imagine a teacher shows a student a map of the U.S. with the states named. Then later, the teacher shows the student a picture of a given state outline, and asks the student, "What is the name of that state?" If the student replies with the correct name, then that is recall, because the student used the exact words and images that were given to them earlier by the teacher. Our observer with the clipboard, sorry… tablet, notes it as such.
If the teacher then asks, "What are 'states'", and the student replies with a list of the names that they remember, or can draw rough outlines of the shapes of different states, then that is also recall.
Recall is repeating what was previously read, seen, heard, or demonstrated.
However, if the student responds instead with a story, and can explain the idea of "states" without using phrases or images that were already given to them by the teacher (or someone else), then our observer will note it as demonstrating comprehension.
For example, imagine that the student is now observed explaining "states" to a younger student. First imagine that the student simply repeats what they were previously shown by the teacher: a labeled map. But then imagine that instead, the older student acts more like a parent (instinctively, many parents actually "get" this part), and tells something like a story: "In a big country like the U.S., people decided to separate out different areas to make it easier to manage. Sometimes they used a river or some mountains to decide where to make the lines that divide the areas. Sometimes they just drew a line on a map. The different areas that they ended up with are called "states". Notice that in the second case, except for the concluding sentence at the end, none of the words or images that the teacher originally gave the older student are used in the explanation. This is the one definitive demonstration of "understanding" or "comprehension".
Comprehension is evidenced by re-expressing, without using what was already read, seen, heard or demonstrated.
Why? Because it's impossible to correctly re-express a concept without actually comprehending it. And in fact, it is equally impossible for the teacher, or our objective observer, to know whether the re-expression is correct unless they themselves also comprehend the concept.
Important point: If the teacher uses the above explanation about large countries, rivers, etc. to describe "states", or if the student is given the same or alternate explanation by someone else (parent, another student, Internet resource, etc.), then that particular explanation is then excluded from a response that would qualify as comprehension! Why? Because the use of the explanation gained from another source would then again just be repeating, and not re-expressing. That is, it would just be recall.
This is the reason that a professional educator is required for there to be the most successful learning environment possible. By professional educator, it means someone who has been trained to be aware of both best practice pedagogical techniques for instruction, who actually understands the concepts that they are teaching so as to be able to correctly assess the accuracy of the re-expression, and also to maintain an ongoing awareness of what "answers" have already been given to the learner so as to be able to specifically exclude those pre-given answers to any later measure of whether comprehension is being demonstrated.
Now imagine that the teacher asks the student to take a topographical map of a region that has not been divided into "states" (or countries for that matter), and to use what they've learned about the concepts of how states and countries are delineated, and to make their own new map. This would be an example of application, because although the possible solutions (rivers or mountain ranges) are known to the student, the particular problem itself (the given map) does not yet have a known solution. The student can only create the application example if they have an underlying comprehension of the concept, and demonstrates it for a problem that they have not previously seen.
Assignment: Use what you've learned about the borders for countries and states to create your own borders in this map.
Are there historical examples that you can find of where borders were specifically drawn? How effective were the borders afterwards, and by what criteria did you assess their effectiveness? Trade? Culture? Conflict?
Application is the process of matching the new problem given with an available inventory of known solutions. For that it will be necessary to re-express already-known solutions to the given problem. Application is solving a new problem for which the solution to other similar problems is already generally known.
For synthesis, the teacher could now present a challenge, or the student could encounter it on their own, where it would be necessary to take what was leaned about dividing physical areas into regions, and combine it with some rather different conceptual knowledge, such as where different kinds of animals and plants are found (and before the students know about biomes), and defining the "borders" between what they have decided will be different zones.
Assignment: Make a map of the world, your continent, country or even local region, and draw on it to indicate where think different types of animals and/or plants are found.
Then use what you've learned about the borders for countries and states to create your own borders of plants and/or animals and think of some creative names for your new "countries" of living things.
In this example, the student must consider what they have learned and applied with known solutions, and then essentially "invent" new criteria for delineating "countries" of plants and animals. I could give examples here of what those might be, but in giving the examples, I would eliminate those from the possible solutions that would qualify as synthesis.
Synthesis is solving a problem for which the solution is not already known (not already given to the student by the teacher or another source), and which requires bringing together multiple solutions to create something that clearly didn't exist before. "Intention" is an important aspect and is implied by the objective of solving a problem. The problem itself need not be independently initiated by the student, but the solution(s) to the problem do need to be such that they have independently put them together for solving the problem. If the student "looks up" the answer, or simply gets "the answer" from another source (friend, parent, Internet search, etc.) then that invalidates that response as demonstrating synthesis.
This may seem to be a harsh criterion, but it is a realistic measure of whether true synthesis is taking place. For example, if the student is presented with a topological puzzle ("Can you connect 9 dots with 4 lines?"), and solves the problem through their own experimentation and insight, then that would demonstrate synthesis.
But if a friend just shows them the "trick", then repeating it is simple recall.
Thus, if the challenge is to create a timeline of historical events, and the student already knows how to order events by date, and is familiar with the concept of timelines, then the creation of the timeline is at best application, not "synthesis", or being "creative" in a meaningful way.
Synthesis (real creativity) is solving a problem with combined solutions that are not already known.
Here is a more concrete example with older students (and their professors) presented with what sems to be a simple challenge, but who then encounter difficulty.
The video is procatively titled, "MIT graduates cannot power a light bulb with a battery."
This video has much food for thought, and can be easily understood through the model presented here for Bloom's Taxonomy.
Here are some key quotes from the video,
“You can’t always assume that what is blatantly obvious to you, and what has always been blatantly obvious to you, is going to be that way for somebody else, especially a kid. Is this really as self-evident as you think it is” - Jim Carter, Physic Teacher
We don't know whether the teacher was presented with the same problem and figured it out on his own, or watched the video and then commented on it, but my guess is the latter. And he even hints at the real answer in saying that one can't assume that the answer is obvious to someone else, "especialy a kid".
The insight to be had here is that regardless of the age of the person trying the challenge, if they haven't already been shown the answer, "synthesis" is required to put together what it already known about conductive circuits, batteries and light bulbs, and come up with an "answer" that wasn't previously known.
The video also has some other important statements.
“If one cannot light a light bulb with a battery and one wire, then everything built upon those basic ideas has problems.”
This statement is not quite correct. One can understand the concepts of circuits, batteries and light bulbs without having been presented with this particular challenge. The activity of the video really is a "trick question", in that they chose something that was not in any students' common experience. And, it can be noted, that we don't know how many students were able to solve the puzzle that weren't included in the video.
It is quite possible that the students could have answered any other question about electricity from their prior memorized instruction, and also quite possible that overall, their understanding of electricity was still limited.
At brilliant.org is a page that discusses the video:
This commentary seems to take the position that what was lacking in the student and their professor's failure to meet the challenge was their having forgotten the basics of electricity, batteries and light bulbs.
It's true that they may have forgotten the basics, but the telling point is this,
"The obvious way to close the circuit and light the bulb is to use two wires."
The really is the point. The obvious way is the one that any student should be able to reproduce from memory (recall) even without understanding why (comprehension).
However, to come up with the non-obvious solution requires something very special: synthesis based on recall, comprehension and application combined in a new way for the first time (for the learner).
Synthesis isn't easy or common. That's why it's so elevated in what's hoped for in assessment. And, as we've already seen, the second time the solution is offered, it's only recall.
One last comment before moving on from the discussion of this video is related to this statement by the narrator,
“Sometimes, the simplest problems in science defy intuition, and the most basic technologies are surprisingly difficult to grasp.” - video narrator.
This has a lot to unpack because for some natural phenomena (and I avoid the word "science" here), they do defy intuition, and transistor is a technology that isn't trivial (perhaps not even possible) to understand completely.
Still, for the subject matter of this video, there is nothing that defies intuition, nor is difficult to grasp. The solution is quite understandable once you see it. Just like the puzzle of connecting 9 dots with 4 lines.
Note: I said I avoided the word "science", and the reason for that is that the word science has for many people, including educators, come to mean "a magic trick performed with natural phenomena". "Science" is a process for making informed (and sometimes not so informed) guesses, testing, and arriving at an explanation for natural phenomena. It is not just watching something change colors or make smoke.
The Circular Nature of the Phases of Learning
Simply put, this is just the observation that no matter how clever and innovative the re-expression is that demonstrates "comprehension", once the learner has used it themselves, the next time they use it, they will be back at recall. We have all had the experience of coming up with a new way of explaining something that particularly pleases us, and then going back to that explanation in future years while teaching that class.
Even though the first time qualified as "comprehension", and demonstrated a new insight and understanding of the contenton our own part, the later repetition of that is simply recall. It doesn't mean that the comprehension is lost, only that from the standpoint of an impartial observer, they don't know years later whether the teacher (or student in adult life) still retains the comprehension of the concept, or is simply repeating what they remember as the correct explanation of something, even if they did (back then) come up with that explanation themselves. Only by demonstrating new re-expressions of the concept (or demonstrating new instances of synthesis) would our impartial observer be able to say that they had continued to demonstrate that their comprehension of the concept had been retained.
The rationale behind the cycle is that even if the solution to a problem wasn’t known to the learner the first time a new problem was encountered, and thus qualified as a demonstration of comprehension, the next time the problem is encountered, the previously “new” solution would be remembered from before, and once again be recall. In fact, even the highest level of synthesis, having been done once, itself becomes recall when the learner uses what was learned previously and demonstrated by “synthesis” in a repeat of the demonstration. In fact, the overall learning process of life is one where the “revelation of learning” at one age becomes the “of course I know that” at a later age.
Likewise, for someone that invents something totally new, some extraordinary demonstration of synthesis, once that solution to a problem is known, repeating it just reverts to recall. It is only the further elaboration and improvement (for example) that would demonstrate further examples of synthesis. Examining Sample Lessons Let's look at the following series of examples of technology-using student-made activities and projects, and de-construct it with respect to the four tests being applied separately to technology and content.
TPACK: Technology, Pedagogy, Content Knowledge
"Any sufficiently novel use of technology is indistinguishable from real learning" - Roger Wagner paraphrasing Arthur C. Clark
See a good discussion of TPACK here: http://mkoehler.educ.msu.edu/tpack/what-is-tpack/ and http://www.tpck.org
Teachers, parents, administrators and even the students themselves make assessments every day as to which categories in Bloom's taxonomy different responses or activities can be ascribed. A big challenge is that the four tests can be applied to the use of the technology processes used in a project as well as the content itself.
That is to say, it's quite easy to find instances where someone will say, "The student was creative, and demonstrated higher-order thinking skills because they made a movie!". However, there are two parts to this: a) the skills required to "make a movie", and b) the demonstration of the learning that took place with respect to the content. Quite often, students, parents, teachers, and even administrators mistake what might even be a legitimate demonstration of "synthesis" with the technology for being a demonstration of synthesis regarding the content . Because newer technologies are, well, "new", their use does often seem to satisfy the application and synthesis tests presented here, and in the process, distract us from asking the same questions about the content.
Do We Need Technology?
Technology is an amplifier, and its purpose is to provide the most enabling, least restrictive environment to increase the possibility and likelihood that the higher levels of learning take place. It does not in itself though, automatically bring into being these higher levels without proper pedagogy, and also without a very large foundation in each of the lower levels to make the higher levels possible. That is to say, a huge amount of recall is necessary to enable a certain amount of comprehension, and a lot of comprehension is required for application. True synthesis is the most difficult of all, and actually rarely happens as one gets older.
What is the Ideal Use of Educational Technology?
In short, the best educational technology should be that which is the most enabling, the least restrictive, to the learning process, and goals of demonstrating comprehension, application and synthesis. Technology that limits the student in their ability to gather and organize information, and later to express themselves and represent their research and thinking, is much less effective.
Ideally, the technology environment should function as a truly enabling tool, presenting as few roadblocks as possible, and in fact leading, encouraging, and permitting the student to do more research, organization and self-expression than would otherwise be possible.
The learning cycle in young children, and why it's understandable for it to diminish with age.
The leap from application to synthesis can happen very quickly for simple concepts and/or clever students, or more slowly in other cases. Experimentation is an important ingredient, because if the solution/answer/process is not known, some degree of experimentation on the part of the learner is always required. It may be visible, physical experimentation on the part of the learner, or it may take place solely in a mental space, as the learner imagines the possible outcomes. By the way, some of the most famous examples of this are Einstein's thought experiments.
For a young child, the process of experimentation, analysis, and evaluation take place almost simultaneously with the initial presentation of the information, because there are very few "already known" solutions to problems. As they age to become adults, people increasingly rely (as they should) on already-known solutions. This is faster and more efficient, and does explain why adults are less explorative and experimentive. It's not necessarily a bad thing, and it's also why many scientific breakthroughs are made by young scientists in their early 20s, or sometimes even younger.
It's very true that the recall of solutions is much faster than the process for synthesis. If you see a car coming at you, and you jump out of the way, because you learned to do that early on, it's more efficient, and has a higher survival value than pondering other possible solutions which are as yet unknown to you. Of course, when there is no known solution to anyone, making recall impossible, such as what to do about a growing ozone hole over the poles, or how to make a practical working light bulb in 1878, then true synthesis is the only way to create a solution.
Note: To get an insight into just how much of Edison's light bulb was "new", and how much was matching known solutions with a given problem (application), see this description of the history of the incandescent bulb: http://en.wikipedia.org/wiki/Incandescent_light_bulb#Early_pre-commercial_research
It's understandable then, that for a very young child, everything is a huge swirl of recall, comprehension, application, and synthesis, and Piaget and Papert describe this process through the models of constructivism and constructionism. However, as people get older, they are able to (and desirably so) act quickly and efficiently on a growing body of remembered behaviors. In fact, the huge body of this knowledge is what gives values to "the elders", who can often say, "I've seen this before, and I can tell you what's going to happen."
On the other hand, in a world that is now changing more and more rapidly, the ability to extend the skills of application and synthesis into later years of life is essential for job security, productivity, and even the long term viability of a democracy. Bloom said it himself back in 1956:
Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain Anderson, Lorin W. (Editor)/ Krathwohl, David R. (Editor)/ Bloom, Benjamin Samuel (Editor), David McKay, 1956
Conclusion: As you first start to try to apply these four tests (assessment tools), it may not seem that easy for you to separate the technology from the content, or to easily classify activities into Bloom's categories. However, with practice, you'll not only get better at it, but find that the rules themselves make this process easier than with any other alternative system. In fact, ultimately I think you'll find it easier and more effective to practice getting better at this model:
than this one:
Also, it's worth noting that your students themselves will do better in their projects, and thinking about their own thinking (metacognition), if they have this same illustration to refer to, and if you discuss the principles witht them on which they will be assessed.
I would greatly enjoy hearing from you in the future as you explore this, both with challenges, and any comments you have about whether this has helped improve your classroom teaching experience.
Postscript: Valuing "creativity" (or synthesis) over all other activities.
Because "create" is at the top of the revised Bloom's taxonomy, and because "creating" can be so easily applied to an activity, there has been a marked tendency to emphasize this as a classroom activity and objective at the expense of the other categories.
The problem is that recall is necessary for comprehension, comprehension for application, and application for synthesis. In fact, you can think of it rather like a "food pyramid":
You can't re-express something without having a large vocabulary of other words (that you already comprehend) with which to do the re-expression. Application is the process of matching the given problem with the available inventory of known solutions. For that it may be necessary to re-express the given problem to match a known solution.
For Synthesis, a huge number of previously known solutions need to be available for combination to result in a previously unknown solution to a challenge, and resulting in a new and unique product.
However, for each of the elements, the processes of analysis and evaluation are continuously in operation.