Coding for Level of Difficulty:
For a full explanation, see A Rubric for Coding Problem Difficulty, from: Renninger, K. A. & Feldman-Riordan, C. (in preparation). "Technology as a tool for developing students' mathematical thinking." (The help of Crystal Akers and Alice Henriques in clarifying this coding scheme is gratefully acknowledged.)
Coding of problem difficulty focuses on the mathematical challenges represented by the problem, the difficulty of the mathematical concept, and the difficulty of mathematical calculations for students at a given level of problem solving. The rating scale consists of 5 levels of difficulty, wherein a Level 5 problem is a very difficult problem for students in a given grade band.
Level 1. Only one concept needs to be worked on; the mathematics is rudimentary and represents prior knowledge rather than something new. Example:
Use a variable to explain an interesting relationship that can be found between three consecutive integers.
The problem is a straightforward application of multiplication of binomials.
Level 2. Either a) the concept is clearly stated within the problem, and the mathematics is challenging for students at the given level of the PoW, or b) the concept requires some "stretching" for students at this level, and the mathematics is based on prior knowledge. (Note: Problems that require attention to explanation are likely to be found at Level 3, rather than Level 2, because of the difficulty involved in explaining mathematical understanding.) Example:
Indiana Jones found a math problem about the life span of "The Great One" engraved on an old tombstone. How old was the Great One when he died?
The concept is clearly stated in this problem, but the mathematics requires slightly more work than that expected for a Level 1 problem. The problem does not contain a twist.
Level 3. The problem (a) contains a "twist" or additional problem requirement that students in this grade band may overlook even though they can complete the problem accurately, and (b) requires discourse knowledge of mathematical concepts and basic mathematical ability appropriate to students at this level. (Note: At the elementary level, multiple parts within a problem make what may initially appear to be a Level 3 problem into a Level 4 problem.) Example:
The product of any four consecutive integers, increased by one, is always a square number. Give at least three instances of that statement and prove that this will always occur by finding an algebraic expression for that "square number."
The first part of the problem is fairly simple, but extrapolating from the patterns to write a generalized equation is a challenge for students in this grade band. Some students may overlook the need to actually prove that this is the correct expression.
Level 4. The problem includes the elements listed in difficulty Level 3 and contains an algorithm new to students in this grade band; students may miss the problem by getting bogged down in the math but not by missing the concept; students may not finish the problem or may not attempt all parts of the problem. Example:
A motorcycle stunt person needs some math equation work to calculate her most daring feat: jumping between ramps at high speeds.
This problem requires students to apply what they know about parabolas to a physics-oriented math problem. There is a lot of math here in which students may get bogged down, and there are four parts to the problem.
Level 5. The problem includes the elements listed in difficulty Level 3 and requires discourse knowledge of mathematical concepts and mathematical ability above the level of the students in this grade band; or it contains a concept, theorem, or algorithm that a rater familiar with this mathematics topic does not recognize. Example:
© 1994- The Math Forum at NCTM. All rights reserved.