Coding for Level of Difficulty:
For a full explanation, see A Rubric for Coding Problem Difficulty, from: Renninger, K. A. & Feldman-Riordan, C. (in preparation). "Technology as a tool for developing students' mathematical thinking." (The help of Crystal Akers and Alice Henriques in clarifying this coding scheme is gratefully acknowledged.)
Coding of problem difficulty focuses on the mathematical challenges represented by the problem, the difficulty of the mathematical concept, and the difficulty of mathematical calculations for students at a given level of problem solving. The rating scale consists of 5 levels of difficulty, wherein a Level 5 problem is a very difficult problem for students in a given grade band.
Level 1. Only one concept needs to be worked on; the mathematics is rudimentary and represents prior knowledge rather than something new. Example:
Mrs. Bender noticed that the total of the items her children purchased was $155. Can you determine the cost of one pair of jeans?
This problem requires that students subtract the cost of several items from the total to find the cost of two pairs of Devin's jeans, then divide the result by two to get the cost of one pair of jeans. Subtraction and division should be prior knowledge for students in the middle school level grade band. The most difficult part of this problem is sorting through the story to find the pertinent information.
Level 2. Either a) the concept is clearly stated within the problem, and the mathematics is challenging for students at the given level of the PoW, or b) the concept requires some "stretching" for students at this level, and the mathematics is based on prior knowledge. (Note: Problems that require attention to explanation are likely to be found at Level 3, rather than Level 2, because of the difficulty involved in explaining mathematical understanding.) Example:
Ms. Nelson claims that she is three times as old as her little sister. However, six years ago she was five times as old as her sister was. How old is Ms. Nelson?
This problem could be solved by the guess-and-check method, which should be prior knowledge for the student and would ordinarily cause this to be scored as a level 1 problem. The requirement to explain the process and reasoning behind the chosen method, however, stretches the student's mathematical skills, boosting the problem to Level 2.
Level 3. The problem (a) contains a "twist" or additional problem requirement that students in this grade band may overlook even though they can complete the problem accurately, and (b) requires discourse knowledge of mathematical concepts and basic mathematical ability appropriate to students at this level. (Note: At the elementary level, multiple parts within a problem make what may initially appear to be a Level 3 problem into a Level 4 problem.) Example:
Explore the game of tic-tac-toe using a cylindrical tic-tac-toe board.
The level of discourse and basic mathematical ability required by this problem are appropriate for students in this grade band. The main difficulty the problem represents involves clearly explaining why one player can always win.
Level 4. The problem includes the elements listed in difficulty Level 3 and contains an algorithm new to students in this grade band; students may miss the problem by getting bogged down in the math but not by missing the concept; students may not finish the problem or may not attempt all parts of the problem. Example:
Find the total cost of purchasing tiles for my office.
The concept behind this problem is easily within the range of ability for students in this grade band, but the problem contains smaller components that students can overlook or answer incorrectly; for example reporting the cost of individual tiles rather than the cost of a complete box of tiles.
Level 5. The problem includes the elements listed in difficulty Level 3 and requires discourse knowledge of mathematical concepts and mathematical ability above the level of the students in this grade band; or it contains a concept, theorem, or algorithm that a rater familiar with this mathematics topic does not recognize. Example:
This two-week learning experience challenges students to find information about Pick's Theorem and to solve an area problem using this method.
This problem requires that students learn about a new concept, Pick's Theorem, and understand it well enough to apply it to the problem. The problem also includes many smaller questions that students might overlook.
© 1994- The Math Forum at NCTM. All rights reserved.