T-Tests, P-Values, and Statistical Significance
Date: 03/18/2004 at 22:15:49 From: Matt Subject: What does the t test tell you? I am doing a lab report comparing two different samples of fish. For the results the teacher wants a t-test. I have calculated it and lots of other information. Could you tell me what this information means and how I interpret it? What does the t-value and two-tailed P-value tell me and how do they compare to each other? Is this information "significant" enough to say that variable 2 came from the same family as variable 1? Here is my data: Variable 1 Variable 2 Mean 0.562 0.09152 Variance 0.00097 0.003081962 Observations 5 5 Pooled Variance 0.002025981 Hypothesized Mean Difference 0 df 8 t Stat 16.52697943 P(T<=t) one-tail 9.06766E-08 t Critical one-tail 1.85954832 P(T<=t) two-tail 1.81353E-07 t Critical two-tail 2.306005626
Date: 03/19/2004 at 09:24:34 From: Doctor Achilles Subject: Re: What does the t test tell you? Hi Matt, Thanks for writing to Dr. Math. A t-test tells you the probability that two sets of values come from different groups. Using a one-tailed P-value assumes you already know before you even see the values which group should be larger and which should be smaller. Since this is usually not true, you should almost always use a two-tailed test. Let's say, for example, that I have the following hypothesis: "The average age of trees in Yellowstone National Park is significantly different than the average age of trees in Yosemite National Park." How would I test that (let's assume that I have a way to accurately determine the age of a tree without cutting it down)? I don't have the resources to check the ages of all the trees in each park, so I will take a small random sample from each park and then use a t-test to compare them. A two-tailed P-value of 0.6, for example, would mean that there is a 0.6 (or 60%) chance that the two sets of values come from the same group. In other words, there is a 60% chance that the average age of the trees in each park is the same, and that whatever difference I may have seen in my random sample can be explained by the fact that I only sampled a small portion of the trees. If I got a P-value of 0.6, I would say that there is no significant difference between the ages of the two populations. A two-tailed P-value of 0.1 would mean that there is a 0.1 (or 10% chance) that the two sets come from the same group. In this case, there is a pretty good chance that the ages of the two populations is different. However, in order to be on the safe side, it is traditional in science to say that a P-value of 0.1 is NOT significant. Why? Because if 0.1 were considered significant, then 10% of all scientific findings would be false. So even if I got a P- value of 0.1, I couldn't say anything for sure, the most I could say is that more study is probably required. The traditionally accepted P-value for something to be significant is P < 0.05. So if there is less than a 5% chance that two sets came from the same group, then it is considered a significant difference between the two sets. A t-test computes a "t-value". There is a complicated mathematical relationship that I don't know off the top of my head between a t- value and a P-value that depends on the size of the samples (and one or two other variables). Larger t-values translate into smaller P- values. So the larger the t-value is the more likely the difference is significant. A "critical t-value" is the minimum t-value you need in order to have P < 0.05. If your t-value is greater than or equal to the critical t-value, then you will have a significant difference. In your problem, your critical t-value is 2.306005626, your t-value is 16.52697943. So, your t-value is greater than the critical t- value, therefore the difference between the two sets is significant. This is confirmed by the fact that your two-tailed P-value is 1.81353E-07 or 0.000000181353; this is extremely small (much less than 0.05). I would call the difference between these values "highly significant". (If I were you, I'd go back and recheck that you entered all the values correctly, since this difference is much more significant than one usually gets with only 5 samples in each set; but if you did, then congratulations: you have found a big effect of whatever you were testing!) I hope this explanation is helpful. If anything is unclear, or you'd like to talk about some of this more, please write back. - Doctor Achilles, The Math Forum http://mathforum.org/dr.math/
Search the Dr. Math Library:
Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.