Paul, I thought about sending this response directly to you, but I think its appropriate for statistics teachers to know a little about an alternative method of conducting research. I would NEVER argue with you about quantitative research. I have too much respect for your knowledge and expertise (I am currently teaching with ActivStats and think its the greatest tool ever invented!!). However, there are some things about qualitative research I think you should know. First, qualitative research addresses questions that cannot be answered quantitatively. For example, in my dissertation, I have asked: "How do projects, involving data gathering and group work, effect students' learning of statistical concepts?" The assumption is that projects DO make a difference, but what is it? Is it positive, or did it actually take away from their learning? In order to answer this question, I gathered bi-monthly project reports, audio taped students weekly group meetings, attended and taped their oral presentations, obtained copies of their final project reports, and probed certain items during individual interviews. I am using a qualitative software package to assist in organizing and analyzing this data (which is ALL WORDS). In order to gather these materials, I had to select the project groups to follow in detail. I simply could not manage this much data for all 12 groups. This process is NOT random, but is defined in qualitative research as! "purposive". In other words, "informants are selected as salient features emerge". I spoke at length, many times, with the instructor (a College Board consultant for AP Stat) about who should be selected. I was interested in gathering a cross section of academic talents, but also was concerned that selected students would be open and receptive to my probing. I purposely chose students whom I felt would be most helpful/informative in my attempt to address my research questions. There was nothing random about it and qualitative studies are designed to accomodate this non-randomness. (I have tons of literature to support this.) As I stated in the response to Peg, qualitative reseachers state limitations and conditions that created the selected informants. So I stand by my original claim, provided her study is qualitative. If her research is quantitative, your explanation of voluntary bias was excellent. However, in my dissertation, I will include responses from a survey I put to the listserv about 2 months ago as part of a holistic description of AP Stat. Part of that description includes the listserv - the history, purpose, participants - and I will state in very clear terms, that there is voluntary bias in the survey results. I am NOT stating all AP Stat teachers would respond in the same way. Qualitative researchers do NOT try to generalize to the population. Some things I find may NOT be worth a dang to another teacher (and might be informative to others). However, we believe you CAN describe learning and the teaching/learning process without the same rules that define quantitative res! earch. The qualitative research process is rigorous, valid, and respected in Math Education research (for example, Educational Studies in Mathematics, Journal for Research in Mathematics Education, just to name two of the more prestigious journals). There may be places where your definition of a qualitative study is correct. I don't claim to be an expert on qualitative research - that's what my dissertation committee is for. But one thing they have stressed to me is this. Qualitative studies are alot more flexible than quantitative - not less rigorous, just more adaptable to the individual research site. There are many who believe there is much to be learned by examining individual cases to see what is there. It may not say a little about "everybody" but rather alot of detail about a small segment. I wasn't out to prove or disprove anything. My research purpose was to investigate the learning experience in an AP Stat class, taught by an experienced teacher whose pedagogy is exactly what the Development Committee recommends. I believe the listserv is an important resource for novice AP Stat teachers, and I will include survey results as information that some listservees chose to respond. Each reader will decide w! hether the information is relevant to their situation or not. Just as a sidenote, the teacher I observed had the best scores in the nation (35 fives and 5 fours, no scores lower than 4). so just maybe, I will find some things out that can help some (not all) teachers.
-----Original Message----- From: Paul Velleman [SMTP:firstname.lastname@example.org] Sent: Wednesday, January 14, 1998 5:10 PM To: email@example.com; Kim Robinson Cc: firstname.lastname@example.org Subject: Re: qualtiative study
Sorry Kim, but it just aint so. Voluntary response data are *worthless*. One excellent example is the books by Shere Hite. She collected many responses from biased lists with voluntary response and drew conclusions that are roundly contradicted by all responsible studies. She claimed to be doing only qualitative work, but what she got was just plain garbage. Another famous example is the Literary Digest "poll". All you learn from voluntary response is what is said by those who choose to respond. Unless the respondents are a substantially large fraction of the population, they are very likely to be a biased -- possibly a very biased -- subset. Anecdotes tell you nothing at all about the state of the world. They can't be "used only as a description" because they describe nothing but themselves.
My understanding of the term "Qualitative study" is that it refers to information collected in a setting where quantitative measurements are not feasible, but in which sound experimental or sampling procedures have been used. I would have no objection to a survey of AP Stats teachers that observed their work in class and judged how successfully they taught, say, the Central Limit Theorem -- provided the classes observed were selected by appropriate random sampling and observed following a planned observation protocol by trained observers. That would, by my understanding, be a qualitative study.
And Peg, I certainly didn't mean to come down hard on you; sorry if it came across that way. I just hate to see people spend much time an effort on studies that are doomed from the start and that could easily be made much more valuable. I see too many of these, and my frustration leaks through at times.
> >Paul, > wow, I am really going out on a limb here, but I disagree with >your comments. IF she is doing a qualitative study, describes the >listserv (purpose, participants), and discusses the bias, it is completely >acceptable. The results would be used only as description and would not >be generalizable or transferable. But qualitative researchers appreciate >the unique cases as long as they are stated as such. Your comments are >certainly true if she is intending on conducting a quantitative study. >Did I just open a can of worms?? > >Kim Robinson >Instructor of Mathematics >Clayton College & State University >Morrow, Georgia > >-----Original Message----- >From: Paul Velleman [SMTP:email@example.com] >Sent: Tuesday, January 13, 1998 9:22 PM >To: Margaret M. Balachowski; firstname.lastname@example.org >Subject: Re: A Graduate Student Investigation > >Peg, >Although I am pleased to see interest in how statistics is taught, I'm >afraid that your posting would make a good lesson for those classes in how >*not* to conduct a study. You are soliciting voluntary responses from an >already biased subpopulation (the AP Stat list). Both voluntary responses >and biased sample frames are deadly -- there is *no* way to recover useful >information from data gathered in this way. I hope that you will discard >all of the responses you get and attempt to conduct a properly randomized >survey. Otherwise the information you collect will be virutally worthless >as an indication of what is really happening out there. > >-- Paul Velleman > > >>Hello Everyone - I am a second year graduate student at Michigan >>Technological University in Houghton, MI. I am in the process of >>researching how statistics is taught at the secondary and >>post-secondary levels. I was overjoyed to find this list!! >> >> >>I hope you will be so kind as to respond to several questions that I >>am investigating: >> >>1. How long has your school been teaching any statistics course? >>(Not just the AP course). >> >>2. What book do you use in your AP course? Is it the same for >>everyone in every school? >> >>3. Who made the decision to teach the AP course? Does your >>school have other AP courses? >> >>4. What software, if any, do you use? >> >>5. Do your students crunch numbers by hand, using calculators >>like the TI-85, or is it all done with software? >> >>6. How many students take the course? Is it semester or full >>year? >> >>7. How large is your school? >> >>8. Do you use any totally innovative methods in your class? >> >>9. Do you use the WWW to access data sets? If not, where do you >>get data? If yes, what sites do you use? >> >>10. Are there any good videos, films, etc that you would >>recommend? Do you use them in class? Do you ask students to do any >>reading (newspapers, magazine articles, etc) and write reports? >> >>11. Do your students do projects? How are they graded >>(evaluated)? Do they do presentations? >> >>12. What other types of evaluations do you use? Quizzes? Hour >>long exams? Lots of Homework? Do you require memorization of >>formulas? >> >>13. What is the biggest change in the statistics classroom in the >>last ten years? >> >> >>Thanks in advance for any responses. I really appreciate it!!! You >>can respond to the list, but please feel free to respond directly to >>me. >> >> >>Peg Balachowski >>MS Candidate >>Math Dept. >>Michigan Technological University >>Houghton, MI 14430 >>======================================================================= >>The Advanced Placement Statistics List >>To UNSUBSCRIBE send a message to email@example.com containing: >>unsubscribe apstat-l <email address used to subscribe> >>Problems with the list or your subscription? mailto:firstname.lastname@example.org >>======================================================================= > > >======================================================================= >The Advanced Placement Statistics List >To UNSUBSCRIBE send a message to email@example.com containing: >unsubscribe apstat-l <email address used to subscribe> >Problems with the list or your subscription? mailto:firstname.lastname@example.org >======================================================================= >======================================================================= >The Advanced Placement Statistics List >To UNSUBSCRIBE send a message to email@example.com containing: >unsubscribe apstat-l <email address used to subscribe> >Problems with the list or your subscription? mailto:firstname.lastname@example.org >=======================================================================
======================================================================= The Advanced Placement Statistics List To UNSUBSCRIBE send a message to email@example.com containing: unsubscribe apstat-l <email address used to subscribe> Problems with the list or your subscription? mailto:firstname.lastname@example.org =======================================================================