Monday, January 21, 2008

Testing for misconceptions in the classroom

I came across this web site when trying to pick a geologic misconception to discuss. It was interesting and I thought I would point it out. It is a 73 question test designed to assess student learning in Earth Science classes. The questions have been carefully designed and modified after many student and teacher interviews.

What specifically brought this to my attention is one of the sample questions they provided. The students had to have a misconception of how geology worked in order to get the sample question wrong.
Sample GCI Question #2. The following maps show the position of the Earth's continents and oceans. The o's on each map mark the locations where volcanic eruptions occur on land. Which map do you think most closely represents the places where these volcanoes are typically observed?
A surprising number of students chose the equator/tropics “because it’s warmer there.” This demonstrates that some students often have misconceptions about how geology works. However, I wonder if this portion of students are those that don’t pay attention in class and don’t do their homework? Any rudimentary discussion on plate tectonics covers the Ring of Fire (which was one of the choices) and so they should be able to answer this question. If these are legitimate “studying” students, why are they getting this question wrong?

Assessment tests like this can be useful in finding out what concepts students are not understanding (especially if they are observed in a large portion of the class) and help make us better instructors. But when would you apply this test? The 73 question set is designed for you to make into a 15 question test. I think I would try this test during the middle of the class to see if there are any concepts I need to clear up before continuing. I don’t think it would replace my normal exams with this test because I don’t agree with a bunch of the questions on the test. It may be because I am over-thinking some of them. For example:

3. If the single continent in #73 did exist, how could scientists estimate the time needed for the single continent to break apart and form the arrangement of continents we see today?

(A) Scientists do not yet have a valid method for estimating the time needed to break continents apart.
(B) Through comparison of fossils found in rocks
(C) Through analysis of carbon in rock
(D) Through analysis of uranium and lead in rock
(E) Through comparison of different layers found in rocks

I am guessing the correct answer is (B) fossils and they are using examples like Lystrosaurus that are found in South America and Africa. However, the rifting phase also generates a lot of igneous rock so U/Pb dating is also possible. While (E) is a little more ambiguous and doesn’t give us time (without fossil or isotope data), it’s exactly what geologists do. We compare layers of rocks and we can compare those in South America and Africa and find similarities that tell us they used to be together.

They designed this multiple-choice test to have answers that are ambiguous along with the correct answer. I find those tests particularly difficult because you can over-think them. You can usually think of a reason to include another answer (especially frustrating if it’s choose all that apply), but you don’t have any way to explain your choices. I think that’s what demonstrates real knowledge, having to explain why you chose those answers. I understand they chose multiple-choice for ease of grading, but I think tests where students have to demonstrate their knowledge are better (but much more time consuming to grade).


Kim said...

I think this test is designed, not for grading, but for assessment - for testing whether the course was effective, or whether a new teaching technique was effective.

I just ran across this test as well, while looking for things to use as pre- and post-tests (for intro classes, and for the major as a whole). I haven't read through this, but I like the fact that they've done a lot of research to come up with the wrong answers. They tried to find out what students are confused about, and how they are confused. I haven't looked through it, though. (It was interesting, though, that most professors seemed to think that the test was too easy, but when they tested it on students, the students didn't do very well - the students really were confused about some of these things, even the ones that seem absurd.)

Mel said...

Kim, I agree that this is an assessment test to see where students are getting confused. They did a lot of research to come up with this test and overall it is wonderful. I looked through about half the test and agreed with a lot of the questions they were asking, but I had problems with a few of them. Fortunately, this assessment test is designed to be cut down to 15 questions so I could easily include only those questions I thought appropriate for what I have taught.

saxifraga said...

Interesting test. Maybe I would use it as a basis for discussion in class or get back to some of the topics during lectures. Maybe applying it half-way into an intro course, but only if I were prepared to spend time on it afterwards.