|
|
RE: [math-learn] FCI and CCI in China #2
Posted:
Jan 27, 2012 1:33 PM
|
|
|
|
The normalized gain does not confuse the issue, but rather it is just a
different way to look at the results. It doesn't change the results. The
normalized gain is used to remove the bias of student innate tendency to
understand. It is more sensative to what is done in class because it is one
way of factoring out what students already know. Of course it is not
totally insenstive to prior knowledge because the ability to think increases
both prior knowledge and gain. If you use effect size you also see dramatic
gain. Some researchers have just shown the graphs. But each method of
presentation has its own bias. The papers on Interactive Lecture
Demonstrations show just the graphs for individual questions and the results
are extremely striking with very large gains from either Real Time Labs, or
ILDs as compared to lectures. Looking at the evidence shown at the
beginning of the ILD book, pre instruction is between 5 and 17%, after
traditional instruction the scores rise to getween 10 and 23%, but after the
ILDs they rise to between 70 and 90%. This word was done by Sokoloff,
Thornton, and Laws with one paper being in The Physics Teacher 36: 6, 340
(1977), but there are others in American Journal of Physics.
As to a full blown exam, Eric Mazur showed that students did better on a
"full blown physics final" when he had gotten improved understanding as
measured by the FCI. So of course the reaction was that Mazur could not
create a standard physics exam, in other words he was a fool. But he is a
respected Harvard professor who holds a number of chairs. So why not
contact him and ask him to release to you his "standard" physics exam and
also get one from a colleague and compare them. In research one usually has
to take the claim made by the other researchers at face value, and then do
an experiment to refute their results. Saying they are incompetent may work
in politics, but it only works temporarily in science. Goethe's theories of
color vision trumped other scientists because Goethe was extremely nasty and
had a high reputation. But after his death, the competitors were finally
able to score their points.
As to whether the students who get a 5 on an AP test really understand
calculus, one has to do that experiment to know the result. You can not
assume the result without obtaining it. I am sure that there is a fraction
(maybe even large) of high AP scorers who do not understand calculus, but I
am willing to say I am wrong if the experiment shows something else. We
have found that some graduate students in physics have difficulty with the
FCI, and one would assume that graduate students would have perfect results.
The CCI is not measuring the same things that the AP measures. I suspect
that the AP exam would not be able to measure the ability to transfer as
well as the CCI, but one would need more testing for that hypothesis. The
essence of research is when someone claims that the theory is so good that a
particular result is absolutely true, but the experiment disproves it. This
sort of thing has happened time and again in medicine. Physicians denied
that bacteria were the prime cause of ulcers for many years after some
definitive experiments, and they continued to cut out parts of stomachs.
As to the questions on the FCI or FMCE testing purely rote memorization, the
answers to those questions have been told to students, and they still get
them wrong. Part of the psychology behind what is going on can be found in
the web reference.
http://www.sciencenewsline.com/psychology/2012012017430043.html
Students when given a question couched in everyday language then give an
answer based on their gut feeling and do not analyze. They particularize
the way they answer questions, so they only turn on an analysis appropriate
to the type of question. So when they see the heading proportions, they
create ratios, but when there is no explicit clue they often can't make a
ratio because they don't understand proportional reasoning. So it our
judgement based on what we have seen and the testing we have done that the
concept inventories are not answerable usually by rote memorization. The
reason why they convinced us is that researchers were in doubt until they
gave the inventories. Then they saw that students who had been told the
results, still got the questions wrong. Joe (Edward) Redis, and Eric Mazur
are prime examples of this. They even tried better lectures, but still
didn't get better results.
Let us remember that all exams are just measuring something, and even when
we think the measurement is good, it may not be. So extreme faith in the AP
is not any more reasonable than faith in our tests. There are other issues,
and with time they are being researched. At one time the IQ test was
considered to be an immutable test of intelligence. In other words it
wasn't subject to change for an individual and it was a true measure of
intelligence. But researchers like Feuerstein blew this out of the water by
producing a long term large change in IQ. The Flynn effect also blows this
out of the water.
As to how people think, even the most educated and "intelligent" people use
gut reactions instead of analysis. Everyone does it and neither you nor I
am immune to it. We often draw non rational conclusions because of this
tendency. Lawson found that understanding of concepts in science was much
more difficult for things that could not be seen, and showed that there
seems to be a level of thinking above the top level proposed by Piaget. A
lot of the argumentation here is actually gut feeling, and people tend to
manufacture evidence based on the feeling without looking at evidence. As
part of this process people tend to see correlations that do not exist, and
they unconciously ignore evidence that is contrary to their notions. Global
warming and evolution deniers do exactly that, and ignore the fact that the
experts are convinced. Logical arguments do not change people's minds.
They just harden positions and bring out more opposition. This of course
can be seen in the development of things like the helicentric univers vs the
geocentric universe, or medical advances like antiseptics.
One of the things that IE seems to do is break down the student gut feelings
which tie them to positions which are at odds with evidence. The ILDs do
this by memory reconsolidation. One would assume that a student who has
taken calculus would be able to understand that acceleration does not go to
zero just because velocity goes to zero, yet we have found that this
misconception exists even in students who have had 2 years of calculus (I
have seen it in class). Our position is that to be able to analyze well the
students have to line up their gut feelings with their ability to analyze.
IE uses specialized techniques along with consistent coherent instruction.
So there are basically 2 views here. One can assume that because IE works
better for average or below average students that it also works for the
superior student. But of course that has not been proven. The assumption
that standard instruction works best for the superior student is the other
point of view. It has been acknowledged that it doesn't work well for the
lower 80%. But the superiorty for the top 1% hasn't been proven either.
The only evidence that I can point to are some early papers in JRST cited by
Lawson in his book "Science Teaching and the Development of Reasoning" where
they found that the learning cycle works for all students. The evidence was
the exploration had to come first for the students below the formal
operational level, but concept development (lecture) could come first for
the others. But they found that the formal operational students still
needed an exploration phase.
If someone could actually show that the top 1% benefits more by conventional
instruction I would be happy to consider it. But I would like to make sure
that the ability to transfer the learning was tested. I think the PER
community would agree that the ability to transfer is one of the most
important abilities a student can have. And of course the experiment MUST
have two equivalent groups given very different treatments. I suspect that
IE will do a better job, but I am extremely doubtful that conventional
teaching does a better job. My "belief" here is based on the fact that
education up until now has been done by gut feelings rather than by an
understanding of how students learn and "accommodate" new ideas. In other
words what is brain incompatible for lower students is likely to be
incompatible for all students. Until this experiment has been done, it is
quite obvious that arguments using informal observation will not change my
mind. Can anyone come up with some experimental evidence here? I always
ask for evidence and never get any citations to it!
John M. Clement
Houston, TX
>Jerry said that most students that pass calculus don't even know what
calculus is. I agree with him. I wasn't talking
>about that group and the Jerry wasn't talking about students scoring 5's on
AP exams. I put the bar high enough so that
>any more quibbling would be silly.
>
>
> I have read and studied a great deal of the data John (and
> Hake) have pointed me to. And I have looked, as much as
> possible, at the scant "live" examples of IE that they have
> pointed me to.
>
> My simple request for something real, like showing me top
> students practicing I.E. is born of all that review. The
> "research" John keeps pointing me to is ruined by a
> fundamental (and absurd) flaw in the design of the
> experiment. Essentially, an I.E. class is set up next to a
> traditional class and an inventory test is taken at the
> beginning and end (by each class). And then the results are
> compared. Actually, rather than comparing the results
> directly and absolutely, they are obfuscated by converting
> them to "normalized gain" prior to the comparison. But that
> isn't the flaw I was talking about. The flaw is that an
> inventory test is no substitute for a full blown calculus or
> physics exam. Who would form an hypothesis that IE classes
> are better than traditional classes and then not set out to
> prove it with full blown traditional exams? You can't make a
> claim and then redefine how you test the claim as well. To
> make the claim that IE is better than traditional you have to
> face the music and test it using full blown exams, not
> fragments of an exam.
>
|
|
