Academic Progress Monitoring Tools Rating Rubrics
Please note that the following rubrics are applied to each grade level targeted by the tool.
General Outcome Measures
Psychometric Standards
GOM 1. Reliability of Performance Level Score
Reliability of the performance level score is the extent to the score (or average/median of
2-3 scores) is accurate and consistent.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The reliability estimates for the performance level score (e.g.,
Cronbach’s alpha, test-retest and/or inter-rater reliability) are adequate
and appropriate for the tool.
The reliability estimates for the performance level score (e.g.,
Cronbach’s alpha, test-retest and/or inter-rater reliability) are not
adequate and appropriate.
Reliability of the Performance Level Score estimates were not
provided.
GOM 2. Reliability of Slope
Reliability of the slope is the extent to which the slope of improvement accurately
represents rate of improvement.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The reliability of the slope estimates are adequate for the tool.
The reliability of the slope estimates are not adequate.
Reliability of the slope data were not provided.
GOM 3. Validity of Performance Level Score
Validity of the performance level score is the extent to which the score (or
average/median of 2-3 scores) represents the underlying construct.
Rating
Full Bubble
Empty Bubble
Definition
The validity for the performance level score (e.g., content, concurrent,
predictive and/or construct) is adequate and appropriate for the tool.
The validity for the performance level score (e.g., content, concurrent,
1
Rating
Dash
Definition
predictive and/or construct) is not adequate and appropriate.
Validity of the Performance Level Score data were not provided.
GOM 4. Predictive Validity of Slope of Improvement
Validity of the slope of improvement is the extent to which the slope of improvement
corresponds to end-level performance on highly valued outcomes.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The predictive validity for the slope of improvement is adequate for the
tool.
The predictive validity for the slope of improvement is not adequate.
Validity for the slope of improvement data were not provided.
Progress Monitoring Standards
GOM 5. Alternate Forms
Alternate forms are parallel versions of the measure within a grade level, of comparable
difficulty (or with Item Response Theory (IRT) based, item or ability invariance).
Rating
Full Bubble
Half Bubble
Empty Bubble
Dash
Definition
There are at least 20 alternate forms and mean performance on
alternate forms is comparable.
There are at least 20 alternate forms and alternate form reliabilities are
strong.
There are at least 20 alternate forms but alternate form reliabilities are
not strong and mean performance on alternate forms is not comparable.
Alternate form reliability data were not provided.
GOM 6. Rates of Improvement
Rates of improvement specify the slopes of improvement or average weekly increases,
based on a line of best fit through the student’s scores.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The basis for specifying adequate growth is strong.
The basis for specifying adequate growth is not strong.
Rates of improvement data were not provided.
GOM 7. End-of-Year Benchmarks
End-of-year benchmarks specify the level of performance expected at the end of the
grade, by grade level.
Rating
Full Bubble
Empty Bubble
Definition
The basis for specifying end-year performance is strong.
The basis for specifying end-year performance is not strong.
2
Rating
Dash
Definition
End-of-year benchmarks data were not provided.
GOM 8. Sensitive to Student Improvement
Sensitivity is the extent to which a measure reveals improvement over time, when
improvement actually occurs.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The basis for assuming that the data are sensitive to children’s
development of academic competence is strong.
The basis for assuming that the data are sensitive to children’s
development of academic competence is not strong.
Sensitivity to student improvement data were not provided.
Data-Based Individualization Standards
GOM 9. Decision Rules for Changing Instruction
Decision rules for changing instruction provide guidance indicating to a teacher when
s/he should make a change to instruction.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The basis for establishing decision rules for when changes to
instruction need to be made is strong.
The basis for establishing decision rules for when changes to
instruction need to be made is not strong.
Decision rules for when changes to instruction need to be made are not
provided.
GOM 10. Decision Rules for Increasing Goals
Decision rules for increasing goals provide guidance indicating to a teacher when s/he
should increase the goal.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The basis for establishing decision rules for when increases in goals
need to be made is strong.
The basis for establishing decision rules for when increases in goals
need to be made is not strong.
Decision rules for when increases in goals need to be made are not
provided.
GOM 11. Improved Student Achievement
Improved student achievement refers to the relationship between use of the tool and
increases in student performance on external measures of achievement.
Rating
Definition
3
Full Bubble
Half Bubble
Empty Bubble
Dash
The basis for establishing that teachers’ use of the tool results in
improved student achievement is strong.
The basis for establishing that teachers’ use of the tool results in
improved student achievement is moderate.
The basis for establishing that teachers’ use of the tool results in
improved student achievement is weak. Criteria for meeting the full
and half bubbles were not met.
Evidence that teachers’ use of the tool results in improved student
achievement was not provided.
GOM 12. Improved Teacher Planning
Improved teacher planning refers to the tools’ ability to help a teacher in planning for and
adjusting their instruction to meet student needs.
Rating
Full Bubble
Half Bubble
Empty Bubble
Dash
Definition
The basis for establishing that teachers’ use of the tool results in
improved teacher planning is strong.
The basis for establishing that teachers’ use of the tool results in
improved teacher planning is moderate.
The basis for establishing that teachers’ use of the tool results in
improved teacher planning is weak. Criteria for meeting the full and
half bubbles were not met.
Evidence that teachers’ use of the tool results in improved teacher
planning was not provided.
4
Mastery Measures
Psychometric Standards
MM1. Reliability
Reliability is the extent to which scores are accurate and consistent.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The reliability estimates (e.g., Cronbach’s alpha, test-retest and/or
generalizability theory) are adequate and appropriate for the tool.
The reliability estimates (e.g., Cronbach’s alpha, test-retest and/or
generalizability theory) are not adequate and appropriate.
Reliability data were not provided.
MM2. Validity
Validity is the extent to which scores represent the underlying construct.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The validity data are adequate for the tool.
The validity data are not adequate.
Validity data were not provided.
Progress Monitoring Standards
MM3. Skill Sequence
The skill sequence is the series of objectives that correspond to the instructional hierarchy
through which mastery is assessed.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The evidence for the skill sequence on which the MM system is based
is adequate.
The skill sequence on which the MM system is based is not adequate.
Skill sequence data were not provided.
MM4. Sensitive to Student Improvement
Sensitivity is the extent to which a measure reveals improvement over time, when
improvement actually occurs.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The basis for assuming that the data are sensitive to children’s
development of academic competence is strong.
The basis for assuming that the data are sensitive to children’s
development of academic competence is not strong.
Sensitivity to student improvement data were not provided.
5
MM5. Pass/Fail Decisions
Pass/fail decisions are the metric in which mastery measurement scores are reported.
Rating
Full Bubble
Empty Bubble
Dash
Definition
The basis for defending the pass/fail decisions in the system is
adequate.
The basis for defending the pass/fail decisions in the system is not
adequate.
Pass/fail decision data were not provided.
Data-Based Individualization Standards
MM6. Decision Rules for Changing Instruction
Decision rules for changing instruction provide guidance indicating to a teacher when
s/he should make a change to instruction.
Rating
Definition
The basis for establishing decision rules for when changes to
Full Bubble
instruction need to be made is strong.
The basis for establishing decision rules for when changes to
Empty Bubble
instruction need to be made is not strong.
Decision rules for when changes to instruction need to be made are not
Dash
provided.
MM7. Decision Rules for Increasing Goals
Decision rules for increasing goals provide guidance indicating to a teacher when s/he
should increase the goal.
Rating
Definition
The basis for establishing decision rules for when increases in goals
Full Bubble
need to be made is strong.
The basis for establishing decision rules for when increases in goals
Empty Bubble
need to be made is not strong.
Decision rules for when increases in goals need to be made are not
Dash
provided.
MM8. Improved Student Achievement
Improved student achievement refers to the relationship between use of the tool and
increases in student performance on external measures of achievement.
Rating
Definition
The basis for establishing that teachers’ use of the tool results in
Full Bubble
improved student achievement is strong.
The basis for establishing that teachers’ use of the tool results in
Half Bubble
improved student achievement is moderate.
The basis for establishing that teachers’ use of the tool results in
Empty Bubble improved student achievement is weak. Criteria for meeting the full
and half bubbles were not met.
6
Dash
Evidence that teachers’ use of the tool results in improved student
achievement was not provided.
MM9. Improved Teacher Planning
Improved teacher planning refers to the tools’ ability to help a teacher in planning for and
adjusting their instruction to meet student needs.
Rating
Definition
The basis for establishing that teachers’ use of the tool results in
Full Bubble
improved teacher planning is strong.
The basis for establishing that teachers’ use of the tool results in
Half Bubble
improved teacher planning is moderate.
The basis for establishing that teachers’ use of the tool results in
Empty Bubble improved teacher planning is weak. Criteria for meeting the full and
half bubbles were not met.
Evidence that teachers’ use of the tool results in improved teacher
Dash
planning was not provided.
7