Collecting and analyzing data on students' overall literacy performance is important to the evaluation of both core (Tier I) and supplemental (Tiers II and III) instruction. As part of your data-analysis process, you will want to examine what the data is telling you about instruction in all tiers.
Quality, evidence-based Tier I instruction is at the foundation of an effective RTI framework, and without it in place, it is likely that the need for Tiers II and III instruction will exceed available resources. Thus, using summative data to evaluate Tier I instruction should be considered a critical step in evaluating the RTI process at your campus.
Likewise, it is important that the analysis of summative data be extended to the evaluation of Tiers II and III instruction, and you and your team will want to ensure that this activity is integrated in the decision-making process for students receiving additional support (McDougal, Graney, Wright, & Ardoin, 2010). Disaggregating outcome data for students receiving Tiers II and III interventions will help you identify trends within and across these groups. If there are significant numbers of students in Tier II or III not making adequate gains, you have reason to further evaluate the efficacy of that intervention instruction. In turn, if there are significant gains made by particular groups of students in specific interventions, you will want to identify why those practices were successful and coordinate resources to ensure those practices are implemented across intervention groups.
Once your team has analyzed patterns in the data and identified possible causes for performance gaps, you will want to extend this process to evaluate the instruction being delivered to students in all tiers. You may consider the following questions when evaluating Tiers I, II, and III instruction and addressing performance gaps:
You may want to download a copy of the entire flow chart Analyzing Outcome Data to Inform Action: Questions to Guide the Process.
When making decisions about students within an RTI framework, intervention integrity is a critical factor to consider in the problem-solving process. This pertains to how well tiered instruction is delivered "with regard to content, quantity, and process" as part of your RTI program (Ball & Christ, 2012, p. 237). Another common term used for this characteristic is fidelity. Fidelity of the intervention program refers to the quality of its implementation and the degree to which practitioners "implement programs as intended by the program developers" (Using Fidelity to Enhance Program Implementation Within an RTI Framework, Center on Response to Intervention).
When determining the degree to which students have met yearly goals, it is important to measure and evaluate the fidelity of instruction delivered to those students and to identify any departures or modifications made to intervention instruction (e.g., modified pacing, inconsistency in time allotted or daily schedule). You and your team will need to determine how you can measure the fidelity of implementation and what impact any deviations have had on student learning, and then you can plan for ways to address any gaps in the delivery of instruction within your RTI framework.
As part of your instructional planning, you may consider ways to better measure fidelity of instruction to ensure that students are receiving high-quality, evidence-based intervention instruction before making decisions about students' future needs. There are a variety of tools that can help measure and monitor fidelity. The Monitoring Fidelity in RTI webinar, available on the Center on Response to Intervention site, is one helpful resource for understanding fidelity in RTI. The webinar also discusses specific tools that may help you monitor the integrity of intervention instruction.
In sum, as you engage in data analysis and critical discussion of these issues, you and your team will want to determine Action Steps to address any identified gaps. As part of the cycle of data-based decision making, you will want to ensure that instructional staff collaborate to set program-wide goals for the next year and create a plan to meet those. As the cycle continues, you will use your balanced assessment plan to determine the impact of your instructional plan and the degree to which goals are being met.
This cycle of data analysis and decision making is part of the practice of creating, evaluating, and revising your data-informed plan for improving literacy instruction, as introduced in the Leadership module of this course.
TO LEARN MORE: Extensive research and resources are available on the topic of using data to inform educational reform efforts. These resources can provide guidance as you establish and maintain this process at your campus in a way that best aligns with the needs of educators and students in your particular context.
The Center on Response to Intervention's Implementation & Evaluation section provides information about RTI implementation, implementing RTI with fidelity, and evaluating your RTI framework. It contains a variety of guidance documents and training modules related to these topics.
The English Learner Institute for Teaching and Excellence: Project ELITE has materials available for conducting BOY (beginning-of-year), MOY (middle-of-year), and EOY (end-of-year) structured data meetings. The EOY materials provide guidelines and protocols for evaluating students' overall literacy performance in grades K–3 and planning RTI instruction, with specific considerations for English learners. Click on the 2013–2014 tab under “Resources” to find the Structured Data Meetings materials.
Brown and Doolittle's (2008) practitioner brief titled A Cultural, Linguistic, and Ecological Approach to Response to Intervention with English Language Learners provides guidance in ensuring that the RTI framework is implemented to meet the unique needs of English learners. Practitioners can find guiding questions to consider when evaluating intervention decisions made about English language learners and the integrity of instruction in each tier. It is available on the Center on Response to Intervention site.
REL Mid-Atlantic conducted a webinar, Root Cause Analysis, which provides an outline of root cause analysis and several resources to help educators implement root cause analyses for problems that arise on their campuses. Root cause analysis can be a powerful method for educators to analyze data in new ways and identify the root causes of events. This type of analysis can help you avoid the pitfall of trying to solve the "wrong" problems and missing the root causes of the issues at hand.
NEXT STEPS: Depending on your progress in using assessment data to evaluate overall literacy performance, you may want to consider the following next steps:
Determine how data are being used to evaluate the effectiveness of programs as they are being implemented.
Discuss how outcome data are used to make instructional and programmatic decisions. You might use the third and fourth columns of the Assessment Audit to think through these issues.
Gather and review the intervention materials (e.g., teachers' guide, instructional manuals) to determine if interventions are being delivered as designed.
Determine which staff members have been trained in analyzing summative data and plan necessary professional development to address staff needs in this area.
Determine procedures for gathering and sharing additional data to support valid decisions for all students and student groups, such as English learners.