The purpose of each assessment should be closely related to the instructional decision that will be made using the data from that assessment. The Action Steps in the Assessment component of the TSLP are organized around these purposes and provide a key reminder to educators to keep the purpose in mind when using assessment data. In the medical analogy, to save both resources and time, we would like our doctors to have in mind how they will use the results of tests they order for us.
You and your team can use your Assessment Audit to guide your discussion about the instructional decisions made from the screening assessments administered at your campus. For each instrument, discuss these questions:
- What are the data from this instrument used for? In other words, what decisions will be made based on this data?
- Are the criteria for this decision defined?
- If yes, are the criteria communicated to teachers and other stakeholders?
- Are the criteria implemented routinely?
You can refer to the third column of the Assessment Audit form for this discussion.
What decisions will be made based on this data?
Once screening is completed, the most important decision you and your colleagues need to make is what to do for students who are identified in the screening process as at risk for reading difficulties. This graphic shows a guide to one way of using screening data in an RTI framework:
Students are identified as at risk based on their scores on the screening assessment compared to the expected level of performance for students in their grade level (i.e., norm-referenced). This expected level of performance is called a cut score. Cut scores (which will be discussed further in the next section) are used to identify which students are not at the target level for a particular skill. The severity of the gap between the cut score and a student's score should determine what steps you take next.
As in the example, you may decide to simply monitor students who scored close to the cut score to see if Tier I instruction alone has sufficiently provided the support needed to get back on track for meeting end-of-year goals. For students who fall further below the cut score on the screening assessment, you may decide to conduct additional assessments to determine the students' specific intervention needs. This kind of assessment is the focus of Action Step A3 and is discussed in Lesson A3—Determining instructional needs.
Throughout this process, teachers making decisions about how to intervene with students identified through the screening as needing additional support may want to look at additional data such as results from previous years (when available), English language proficiency, and educational history. Looking at multiple sources of information about a student's literacy achievement and needs is the most effective way to ensure valid data and valid decisions.
The example mentioned above is just one way the decision-making process can work for using screening data. Some schools may do things differently, such as providing intervention instruction immediately to all students whose screening results place them in the at-risk level. There are acceptable variations in the implementation of RTI. The guiding principle is to use reliable data in valid ways to help students avoid or reverse the educational equivalent of heart disease: problems in reading that become more and more difficult to overcome as the years go by.
What criteria will be used for these decisions?
For screeners to be helpful, they have to classify students as either at risk or not at risk. The score that separates "at risk" from "not at risk" is the cut score (mentioned earlier) and is typically established by the publishers of the screening instrument. Some districts decide to set cut scores themselves if they feel the scores based on national norms do not reflect the local student population. The Center on Response to Intervention (2013) cautions, however, that "this process requires a sufficient sample of students and someone with statistical expertise to conduct the analysis" (Brief #2, p. 3).
When implementing universal screening as a part of RTI, many schools find that the screening results identify large percentages of students in the at-risk category. This can occur for several reasons.
Because the goal of screening within an RTI framework is to catch a gap between target skill levels and a student's performance that is just in the making—unnoticed in regular classroom performance—the cut scores tend to be more stringent than lenient. Test publishers, and most school leaders, would rather err on the side of identifying too many students than miss students who need assistance (Center on Response to Intervention, 2013, Brief #2, p. 3). If you do have a significant number of students who fall below the screening cut scores, it will be important to look at individual students and use other data sources, mentioned above, to help determine if they are at risk or if there may have been other issues interfering with either the students' performance or a reliable administration of the assessment.
On the other hand, a large percentage of students scoring in the at-risk category may actually show a true picture—that there really are many students with gaps in key reading skills. This situation points toward a need for improvement in the general education classroom, for example, Tier I instruction in language arts. You may use this information for the purpose of evaluating your literacy instruction program, which is discussed in more detail in Lesson A5—Evaluating overall literacy performance.
In both cases, a school with a large number of students identified as at risk will need to take a closer look using multiple sources of information to make decisions about how to best use its resources to help each student succeed.
More complex than simple cut scores, decision trees show a series of questions or criteria to determine the best route to serving a student's instructional needs. The decision tree might unite the at-risk cut scores for multiple assessments, as described in this report. Or a decision tree may show the use of different kinds of data that inform the RTI process, such as in this locally developed process. One of the main advantages of decision trees is that they can integrate multiple sources of data into the process as they clearly outline the criteria to be used and the possible decisions that can be made based on those criteria. Decision trees can also help in prioritizing and differentiating among students when a large group is identified as at risk.