The Math Forum

Ask Dr. Math - Questions and Answers from our Archives
Associated Topics || Dr. Math Home || Search Dr. Math

Calculating Errors in Measurement

Date: 08/06/2002 at 16:02:20
From: Jeaneen Aldridge
Subject: Significant digits/calculating error in measurement

Dr. Math:

Please help. I've been looking everywhere for the answer to these two 
related questions.  

Suppose you are told that a digital scale measures to the nearest 
0.2kg and asked to find the absolute error involved in using the scale 
to weigh a cat that weighs 7.4 kg. 

It's not clear from the question whether 7.4 is the cat's "true" 
weight or the weight given by the scale, but I'm going to assume that 
it is the weight given by the scale. The way I figured it, the scale 
is in increments of 0.2kg, so the cat's actual weight would fall 
between 7.3 and 7.5, making the largest possible value for the 
absolute error 0.1 kg. However, my friend says the actual value would 
be between 7.2 and 7.6 kg, making the absolute error 0.2 kg.  

The next question goes like this: a schoolroom ruler measures lengths 
to the nearest millimeter. Find the absolute error in using the ruler 
to measure a line of length 0.5 cm.  

Here again, I'm assuming that if you can measure to the nearest 
millimeter, the actual value (or the measured value; once again it's 
not clear which is which) would fall somewhere between 0.45 and 
0.55 cm and the absolute error would be .05 cm. 

My friend agrees with me on this one, but somehow I feel that we are 
both on shaky ground. I need an authoritative answer and explanation. 
Hope you can help. 

Also, could you explain how to calculate the relative and percent 
error in cases like the two given questions, where you don't really 
know the "true" value of the quantity? In both questions above, I've 
assumed that since the "true" value isn't really known, you would just 
determine the greatest distance that the actual value could differ 
from the measured value and use that difference as the absolute error.  
Every place I've looked defines relative error as the absolute error 
divided by the true (or accepted) value. In this case, the true value 
is unknown.  So, would you just divide the absolute error by the 
measured value?


Date: 08/07/2002 at 13:03:49
From: Doctor Peterson
Subject: Re: Significant digits/calculating error in measurement

Hi, Jeaneen.

Suppose the scale is an analog scale that looks like this:

        7.0   7.2   7.4   7.6   7.8   8.0

Your digital scale sort of snaps the needle to the nearest marking; 
so the actual weight might be anywhere in this range:

        7.0   7.2   7.4   7.6   7.8   8.0
                 7.3   7.5

So the absolute error would be no greater than 0.1. I can't imagine 
why your friend would think a cat weighing 7.2 kg could show up as 

If we took the 7.4 kg to be the cat's actual, exact weight, then the 
absolute error would be 0, since the scale would show 7.4 kg. I doubt 
that this is what was intended; rather, you would see 7.4 not knowing 
that that was the exact weight, and would have to determine from your 
observation what is the MAXIMUM possible absolute error based solely 
on the precision of the scale, namely 0.1 kg.

Note that the errors we are talking about are unrelated to any 
inaccuracy in calibration; if the scale were off by 5 kg, the absolute 
error would be 5.1 kg, not 0.1 kg, but we have no way of knowing that.

You're right about the question concerning the ruler, and for just the 
same reasons as above. Of course, a ruler doesn't really measure 
anything; you read it and can often estimate more accurately than the 
markings. Often it is assumed that you can recognize a value halfway 
between markings, so that you could distinguish 0.45 from 0.5, and the 
error would be more like 0.25. But from the wording of the problem, I 
think you are intended to take it just as you did.

>Also, could you explain how to calculate the relative and percent 
>error in cases like the two given the questions, where you don't 
>really know the "true" value of the quantity...

Ordinarily you go by what you see, not by what you don't know, so you 
would divide the absolute error by the actual MEASURED value to find 
the relative error. After all, "true" values are NEVER known!

But just to increase our confidence, let's suppose that the actual 
value is x, and we measured m. Then the relative error based on the 
actual value would be


whereas our calculation based on the measurement would give


How far off are we? For simplicity I'll suppose that m>x, so the 
difference in these two numbers is

    m-x   m-x          1     1           m-x
    --- - --- = (m-x)[--- - ---] = (m-x)[---] = (m-x)^2/(mx)
     x     m           x     m            mx

What is the relative error in our value for the relative error? It 
will be this number divided by the correct value, (m-x)/x:

    ------------ = (m-x)/m

Interesting: the relative error in the relative error is the same as 
the relative error itself! So for example if we found that the 
relative error was 0.1, the ACTUAL relative error would be off by no 
more than 0.1 of this, making it between 0.09 and 0.11: 10% plus-or-
minus 10%. That's close enough for this sort of estimate.

Does that help reassure you that we aren't losing anything by doing 
our calculations based only on what we know?

- Doctor Peterson, The Math Forum 

Date: 08/07/2002 at 20:11:28
From: Jeaneen Aldridge
Subject: Thank you 

Dr. Peterson:

THANK YOU. You cannot imagine how incredibly happy I was to get your 
reply to my problem. I am so relieved to have an answer that makes 
sense - I have been struggling to find an answer to this problem for 
several days now, and I feel as if a big weight has been lifted from 
my shoulders. I teach high school math, and I am supposed to teach 
significant figures and errors in measurement for the first time in a 
couple of weeks. The book I'm using poses these problems with no 
examples and gives several incorrect answers in the key. I've been 
asking people and looking for information... you just don't know how 
happy you have made me. Now I can feel confident when I present the 
subject to my students. You've indirectly helped a lot of people - 
thanks again and know that you have really been a great help to me and 
my students. I really appreciate it.



Date: 08/08/2002 at 08:43:15
From: Doctor Peterson
Subject: Re: Thank you

Hi, Jeaneen.

It always helps to have several good references, so I looked for a 
few good sites on this topic that will either give a good overview, 
or will cover some of the details beyond what you probably have to 
teach. Here are a few (found by using to search for 
"absolute relative error measurement"; there's a lot more):

   The Vocabulary of Error Analysis - Univ. of Michigan 

   Error in Measurements - Introduction to Chemical Sciences,
   James A. Plambeck, Univ. of Alberta 

   Relative Errors - Numerical Methods Tutorial, MathsDirect 

Also, don't forget to search our site for the phrases "relative 
error" and "significant digit"; we have several discussions that 
may help you.

- Doctor Peterson, The Math Forum 
Associated Topics:
Middle School Measurement

Search the Dr. Math Library:

Find items containing (put spaces between keywords):
Click only once for faster results:

[ Choose "whole words" when searching for a word like age.]

all keywords, in any order at least one, that exact phrase
parts of words whole words

Submit your own question to Dr. Math

[Privacy Policy] [Terms of Use]

Math Forum Home || Math Library || Quick Reference || Math Forum Search

Ask Dr. MathTM
© 1994- The Math Forum at NCTM. All rights reserved.