The methods of arriving at conclusions or critical thinking skills can be learned: example the forms of an implication and validity or truth.
On Thu, Feb 24, 2011 at 5:32 PM, Robert Hansen <firstname.lastname@example.org> wrote: > Thanks Dave, I have looked at IRT before and I will have to do some manipulation but I think I could do a derivation based on IRT principles. > > A simple example. We create a test and administer it to 100,000 toddlers (or their parents) and the test has two questions... > > 1. Can you walk? > 2. Can you run? > > When we get the results back we find that the answers are either no to both, yes to walking and no to the running, or yes to both. But never no to walking and yes to running. In this case, running implies walking 100% and furthermore, since we are taking a snapshot in time of the development of walking and running, walking must come before running. Eventually, all of them will answer yes to both questions. > > A math exam with 30 questions covering multiple domains is more involved than the walking vs running example, but appears to abide by the same principles. When I say that question A comes before question B I am saying that a student getting question B correct has a high probability of getting question A correct as well. In fact, depending on the "distance" between the two problems, the probability approaches 100%. So essentially, I am relating problems to each other based on the probability that a correct response on one problem implies a correct response on another. From that angle I could probably translate this inti IRT terms. But this isnt really news, take an exam and find the students getting the hardest 5 problems correct and they probably aced the whole exam as well. > > > >> Robert Hansen wrote (in part): >> >> http://mathforum.org/kb/message.jspa?messageID=7392256 >> >> > The analysis turned out to be very simple. Take an >> individual >> > question, say question #9, and plot the success >> rate for that >> > one question against the overall success rate on >> the entire >> > exam. For example, students attaining an overall >> score of >> > 20% on the exam, they get question #9 right 40% of >> the time, >> > and students scoring 50% overall, they get #9 >> correct 80% >> > of the time, and so forth. Do this for all of the >> questions >> > and you will find that some questions act like #9 >> (easy questions) >> > while other questions (hard questions) act the >> opposite of #9. >> > While students with overall scores of 50% might get >> an easy >> > question (like #9) correct 80% of the time, they >> will get a >> > hard question right only 10% of the time. As you >> move up in >> > overall score on the exam, students start getting >> some questions >> > right very fast while other questions remain >> difficult till >> > the student achieves an overall score of 50%, or >> 60%, or higher. >> >> The following searches will lead you to more about >> this >> method of analysis. >> >> http://www.google.com/search?q=option+curve+IRT-model >> >> http://www.google.com/search?q=item-characteristic-cur >> ve >> http://www.google.com/images?q=item-characteristic-cur >> ve >> >> In these curves, the horizontal axis typically >> corresponds >> to a variable called "theta", which is directly >> related >> (in some montone increasing way that I don't know >> anything >> about) to the test taker's overall score on the test. >> >> The best functioning items are those in which the key >> (i.e. the intended correct answer) follows a "steep >> logistic curve", which happens when the magnitude of >> the second derivative is large at a couple of >> locations. >> >> http://www.google.com/search?q=logistic-curve >> http://www.google.com/images?q=logistic-curve >> >> Dave L. Renfro > >