Case Study: Stages 4 and 5 - Intervention and Evaluation Plan
--Download file--
Background and Additional Error Analysis Notes
Combining the previously gathered information concerning Justin, an eight year old first grade student from Southern Ohio, an intervention plan has been developed to assist him in overcoming his reading struggles. Baseline and error analysis shows that Justin has difficulty in deciphering age appropriate words in written text. His difficulties seem to be limited to reading, as he is a well received and successful student in other academic areas. Specifically, his reading difficulties appear to be unique and random, but his weakest points seem to center around middle consonant sounds and consonant blends, and the letter “y.” More recent observation would add that Justin has difficulty remembering when a vowel should be made “long,” and upon reminders of the effect of a second vowel or silent E, a former correctly identified final consonant sound will be forgotten with the correct vowel. An example is used to illustrate this point.
If Justin is given a word such as “kite,” he will often say “kit,” forgetting the effect of the silent E. When the final E is pointed out to Justin, he will correct himself and say “kie” with a long I sound and forget the final “t” sound. When reminded of the “t” Justin will again say “kit.” This happens consistently with other vowels as well.
A review of a recently scored, classroom administered third nine weeks word attack checks shows a myriad of errors with little pattern. Justin does usually correctly identify the first letter sound, but often, what is stated after that sound appears random. Examples include “liked” for the word “lunch,” “fur” for “flower,” “king” for “clean,” “wit” for “wide” “sip” for “chips, and “san” for “plane.” As noted, the first letter is usually, but not always correctly identified.
Justin has been considered by all his teachers to be a positive student, and has reportedly been enjoyed by all who have taught him. He does well in non-reading subjects, and is considered to be hyper compared to other students. Teacher reports state that this hyperactivity is increasing. B.O.S.S. data confirms that Justin is more often off task than his peers in both math and reading, but he is more often off task in reading.
Mother interview reveals that Justin’s father did not graduate high school and reads at a third grade level. Justin also has three older brothers who did not finish high school.
Hypothesis testing shows that Justin better remembers materials learned in a one-on-one situation than in a classroom setting. Also, if Justin understands the rules governing a part of speech, he will also understand words made up of those rules. For example, when Justin correctly read “gr” and “and” and reported what sounds the two letter combinations would make, he could also read the word “grand,” which is a combination of the two parts. These facets of Justin’s understanding and learning ability will be used in developing an appropriate intervention plan.
Intervention Plan
This intervention will be a changing criterion designed intervention. Justin’s intervention will be held for four weeks on both Thursdays and Fridays. Because of the randomness of his errors, the intervention will be fluid in what is discussed each week. However, the style of instruction will remain the same. In a basic sense, this intervention will involve focused practice covering the areas of need discovered in weekly readings. Justin’s teacher agrees that Justin, like the majority of those with serious reading problems, has a core deficit in phonological processing (Sindelar, Lane, Pullen, &Hudson, 2002). She further agrees with Sindelar et al. that phonological awareness improves with instruction, resulting in improved reading. Justin’s mother reports that she does not know what would be the best method of improving her son’s performance and further reports that she trusts those working with schools to make the best decisions. Justin appears to enjoy working with the intervention tutor and agreed that he would like taking time each week to work on improving his reading abilities. As Justin is only in the first grade, he was simply told that he would continue to participate in the weekly probes and would also focus on learning how to read the words which were hard for him.
Procedures
Each week on Thursday, after collecting the weekly baseline, Justin and the intervention tutor will read from a passage relevant to readings currently being used in class. The teacher will provide these passages. In the event the teacher finds that there are no relevant passages for a week, an “F” level DRA children’s book will be used, as this is the criterion from which the weekly baseline and intervention effectiveness scores are being taken. Justin will read the chosen passage to the intervention tutor who will make notes concerning every error made. For each error, the tutor will provide Justin with specific explanations concerning the rules governing the word which were missed. To assist in mastering the rules presented, other words will be studied which utilize the same rules.
For example, if Justin misses the word “kite” by saying “kit” the rule of the silent E would be discussed, and other words would be studied which contain a silent E. Other such words might include “bike,” “like,” “take,” etc.
Thursday sessions should last between 30 and 45 minutes, depending on Justin’s concentration. Justin usually does good for at least 30 minutes when working one-on-one. On Fridays, a review of the words and rules taught on Thursday will be used to reinforce the newly learned concepts. There will be no passage reading in the Friday sessions, and the Friday sessions will not last longer than 30 minutes.
Due to limitations in the school, the intervention sessions will be held in the school hallways. This means that there will be occasional distractions as students travel in the hallways. Sadly, there is no way to avoid this interference.
Goals
Justin has shown an average improvement of 1.96 percent in percent of correctly identified words for each of the first seven weeks of baseline. His original goal was set at 61.3% accuracy, but the continuous trend towards improvement has caused the need for a restatement of the goal. The mean of Justin’s fourth through seventh week scores was 61.4%. Justin’s scores, though improving, have been extremely variable from week to week. Considering his seven week improvement (trend line) of 1.96 percentage points per week, a greater average improvement following the intervention would show the intervention’s effectiveness. With this approach, an average improvement of greater than 2.0 percent per week would be desired.
However, there are limitations to this method of defining improvement. First, the “F level” reading passages used only contain between 20 and 40 unique words. If an improvement of 1.96 percentage of correct words were to continue without intervention, Justin would be reading with 100% accuracy in less than 23 weeks (Most recent score of 56% + 23 * 1.96 = 101.08). If Justin’s intervention were to be based on his score for 3/3/04 (76.6% accuracy) he would have been expected to achieve 100% accuracy in less than 12 weeks. It has been postulated by Justin’s teacher that his recent trend towards improvement has been partially due to the fact that he has been receiving additional one-on-one attention in reading every week. This attention has been in the form of reading aloud to the intervention tutor, with assistance when needed and questioning and specification of skills in error analysis. This attention by design includes an element of practice. The data shows the possibility that the additional attention has been an intervention in and of itself.
Another possible goal statement would involve increasing the expected percentage of words read accurately. The mean initial three-week baseline score for the 50th percentile of the class was 69.5% accuracy 83.8% accuracy for the 75th percentile of the class. With Justin’s improvement, his fourth through seventh weeks of baseline showed a mean score of 61.4% accuracy. Were these trends to continue, Justin would surpass the average 50th percentile of his class in approximately five weeks and the average 75th percentile of his class in approximately twelve weeks. However, it is clear from working with Justin and discussing his case with his teacher that he is not truly doing as well as the recent numbers suggest. Removing the highest and lowest scores for the fourth to seventh week scores shows a mean accuracy score of 57.5% (verses 61.4% with the high and low scores included). This is a difference of 3.7%, or two weeks of improvement. It could be that some of the recently, randomly chosen “F leveled” DRA probes have been easier than previous probes, leading to falsely high scores suggesting improvement. It could also be that Justin will soon reach an as yet invisible peak in reading score improvement.
Considering these limitations, an effect size may be the best method of showing success to the intervention. Effect size would be calculated using the "standardized difference approach" as follows: [(Baseline Mean – Intervention Mean) / (SD of Baseline)] (Shernoff, Kratochwill, & Stoiber, 2002).
Evaluation Plan
Treatment Integrity
The reading accuracy intervention for Justin will utilize the following protocol to verify accuracy and integrity of implementation. Also, a weekly checklist is provided to aid in assuring all steps of the intervention are followed (See Appendix).
The intervention is to be implemented for four weeks, over a two day period each week (Thursday and Friday, for the convenience of this class). Though the intervention is being done with the student in a one-on-one setting, the simplicity of the steps should prove useful to teachers and aids in both group and classroom settings.
The Thursday intervention will last from 30-45 minutes, and will involve the following tasks. Before the intervention period, the classroom teacher will select a reading passage relevant to materials covered in school that week. If for some reason a passage is not available (due to state testing, special activities, or other circumstances), an “F Level” DRA reading passage will be used. Since the “F Level” DRA reading passages are being used in weekly baseline probes, it is important to make sure a future probe’s passage is not selected. This assures that all probes are novel texts. The passage to be worked with should be approximately 50 words.
The intervention will begin with the student reading the chosen passage to the intervention tutor. Similar to baseline collection, as the student reads, all errors will be corrected so the student will be able to finish the passage in a timely manner. Please note, however, to allow the student ample time to work through any difficult words, and to not jump in unless the student has erred on the word or is not able to come up with a response to the word. Note every error made, including what the student attempted to say was the correct word. Also, before offering the correct answer, prompt the student that his choice was incorrect, as recent performance has shown that he may immediately offer the correct choice.
After reading the passage, the intervention tutor will have a list of all the errors made on the passage. The rest of the time will be spent going over these errors, instructing the student on identifying the correct word. Teach rules where appropriate (silent E, double vowels, consonant blends, etc.) and use similar words to reinforce commonalities in reading. Write down every word and rule taught, so they can be both heard and visualized by the student.
If time is available, drill the student on the words you have reviewed and add new words following the same rule to continue to reinforce the knowledge learned.
For the Friday session, prepare a list of the words which were reviewed on Thursday. This list should include the additional words which may have been used to reinforce rules or patterns taught. The student will be asked to read through these words. Errors will be noted and corrected, reinforcing the learning which occurred on the previous day. Lastly, the student will be given the opportunity to read through the original passage which was used on Thursday. Errors will be corrected as they are made; hopefully, there will be significantly fewer errors, and the student will see his progress and the ease of reading what was just a day earlier, a more difficult passage. Also, hopefully, the new skills will be retained as the weeks progress.
Evaluation of Progress
For the seven weeks of baseline Justin had regular classroom instruction as well as one-on-one practice working with his tutor for the purposes of baseline collection and error analysis. During this period, Justin’s scores never went up for more than two weeks in a row and never decreased for more than one successive week. It should be expected that during the intervention phase, assuming that classroom factors remain constant, that Justin will not show a decrease in scores for more than one consecutive week. If Justin does show a second week of decrease in percent accuracy scores, a reconsideration of the intervention should be considered.
Final results will be evaluated by comparing post intervention scores with pre-intervention scores. Higher percent accuracy means, a positive effect size, and a steeper trend line have all been discussed as possible indicators of intervention success.
References
Shernoff, E. S., Kratochwill, T. R., & Stoiber, K.C. (2002). Evidence-based interventions in school psychology: An illustration of task force coding criteria using single-participant research design. School Psychology Quarterly, 17, 390-422.
Sindelar, P. T., Lane, H. B., Pullen, P. C., & Hudson, R. F. (2002). Remedial interventions for students with reading decoding problems. In M. A. Shinn, H. M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventative and remedial approaches (pp. 703-729). Bethesda, MD: National Association of School Psychologists.
Appendix
Weekly Treatment Integrity Checklist for Reading Assistants.
Thursday:
1_____ Appropriate reading passage selected
2_____ Paper and writing instrument available
3_____ Errors recorded during student’s reading of passage
4_____ Student given a “second chance” before being told correct word during reading
5_____ Rules from errors reviewed with student
a. Additional similar examples included in the teaching
b. Multiple rules taught for each word if necessary
6_____ If time remains, review what has been taught in the session.\
7_____ After session, create list of words reviewed for Friday Session
Friday:
1_____ Word List from previous day’s work available
2_____ Paper and writing instrument available
3_____ Have student read through the word list
4_____ Student given a “second chance” before being told correct word during reading
5_____ Rules from errors reviewed with student
a. Additional similar examples included in the teaching
b. Multiple rules taught for each word if necessary
6_____ Student re-reads original passage
Filed under: EDC 514-515 Academic Assessment for Intervention
Copyright: March, 2004 - David Profitt
