Case Study: CBM Analysis Introductions

Curriculum Based Measurement style data was collected for 24 students in a low level reading group in a first grade classroom in a K-1 school in Southwestern Ohio. This school uses DRA (Developmental Reading Assessment) assessments three times per year, so the probes designed for this study were taken from DRA leveled reading materials. The probes were designed as follows. Level “F” children’s books were selected as they matched the teacher’s criterion goal for reading accuracy for her students for the ending time of this study (April, 2004). The teacher did not want to focus on fluency, so smaller passages were created from the students’ texts, as the students would be required to read the entire passage to obtain an accuracy score. The teacher’s goal was for the students to read with 94% accuracy at the DRA “F” level by April, 2004.

Each of the CBM passages were approximately 50 words in length. Though higher performing students would have little difficulty with this short of a passage, the teacher believed that lower functioning students would be pushed to their frustration level if the passages were longer. Since most of the students in this class had already been identified as lower performing, this was considered the best level for all. DRA rated texts are leveled in part due to the amount of repetition of words within a passage (Beaver, 2001). For this reason, it was decided that for each passage, the total number of unique words would be divided by the total number of unique words read correctly to obtain a percentage accuracy score. An example of the passages is as follows:

I had a lot of messages today. My mom sent me a fax. It was a funny fax. My friend Samina called me on the phone. It was a long phone call. My granddad sent me a letter. It was a great letter. My cousin sent me an email on the internet.

The above is a 52 word passage containing 31 different words. For the above passage, if the child were to read 20 unique words correctly, the child’s accuracy score would be 20/31, or 64.5%.

The probes were administered three times for week one, and the median score was taken for each child. In the case a child was not available for three probes, the average of two probes was taken. The probe was again given one time to each child for a second and third week to show a class wide trend. Probes were administered in the morning school hours on Wednesdays, Thursdays, and Fridays. Students were tested one at a time outside the classroom, at a desk in the hallway. The hallway was busy with students and other teams of students working nearby. There was never a time when the hallway was quiet or empty. The students in this school seemed accustomed to working in the midst of others. The passages were printed in a single piece of paper in a large (18pt) font, to simulate the font size found in the children’s books. The curriculum passages studied in creating the probes were found to contain both serif and sans serif fonts, but the probes created all utilized Times New Roman, a standard serif font. It should be noted that on two occasions, students did not seem to recognize the letter “g” in the probes, even though this letter has been introduced in their classroom reading sources with the same appearance. Before administering the probes, students were told that I was asking them to read a passage out loud so that I could tell how good of a reader they were. They were assured that this was not for a grade, but that it was important that they do their best. They were further told that if they were unable to figure out a word, I would tell it to them and they would be able to continue until they had finished the passage. At the teacher’s suggestion, students who struggled without progress with a word for approximately 3-5 seconds were given the word so they could continue with the passage.

Results
The following graph shows the results of the probes presented with classroom quartile, classroom trends, subject trends, and subject (Justin) scores over the three week baseline period.

The results of the baseline CBM data are mixed, at best. However, discussion with the classroom teacher shows that the results appeared both typical of the class and typical of Justin. For all three weeks, Justin scored below the 25th percentile of the class. For week 1, Justin’s percentile was 4.2, with an accuracy score of 38.4%. For week two, Justin scored at the 15.8 percentile with an accuracy rate of 64%, and for week three, his percentile was 4.2 with an accuracy rate of 42.4%. Justin was the lowest scorer in the class for the first and third week of baseline testing, and was the third lowest the second week of testing. Justin’s percent accuracy scores were more variable across the three weeks than any of his fellow students.

Class wide scores were also variable. Teacher reporting shows that this class is more variable in their progress than many other classes she has had. She commented that students appear to do well some weeks, and regress on others. If one were to look solely at the class median scores, a trend of doing better on the second week’s probe followed by a regression to below first week levels on the third week’s probe would be seen. For the three weeks, the median accuracy percent scores were, 70, 72, and 66.6 respectively. This pattern can also be seen for the first quartile scores, but with even greater variability. However, for the 75th percentile scores, there was no improvement the second week; all subsequent accuracy scores after the first week regressed (87.7%, 84%, and 77.9%, chronologically). Due to the limitations of this study, it will remain unknown if the scores would have been the same if the median of three probes would have been used for the second and third weeks.

Goals
Trend lines for Justin and the class show that the skills of the two are increasing and decreasing, respectively. Where simple probes and statistics 100% accurate, the skills of the two would meet in approximately five weeks. Because of this, goals for Justin will not be built on having him progress at the same rate as the class. Instead, goals for Justin will be based on the average of the three 25th percentile scores for the baseline period. It is the combined goal of the teacher and myself to see Justin improve to constant minimum accuracy rate of 61.3% by April, 2004. Justin’s current three week average accuracy rate is 48.2%, but it should be noted that the teacher felt his second week score (64%) was abnormally high. This goal would show an increase of 13.1% in accuracy. As is the nature of students, these goals may be adjusted or improved if needed.

References

Beaver, J. (2001) Developmental reading assessment: K-3 Teacher resource guide (Rev. ed.).Parsippany, NJ: Celebration Press.

Filed under: EDC 514-515 Academic Assessment for Intervention
Copyright: February, 2004 - David Profitt