Case Study: Stage 1 - Context Definition Teacher Consultation and Probe Creation Introductions and Goals

On January 14, 2004, I met with Mrs. D., a first grade teacher of a lower level reading group (n=23) at a suburban Southwestern, Ohio elementary school. We discussed her participation in a case study and intervention.

This school currently uses DRAs (Developmental Reading Assessments) to chart the reading growth of the students. DRAs are used by teachers in transitional first, second, and third grades to determine reading ability in the areas of decoding and comprehension, using a range of text difficulty. These are similar to, but different from DIBELS or CBMs. As she is a language arts teacher, Mrs. D wishes to focus on reading for this project. However, she does not like the idea of an assessment which looks at frequency, or words per minute. She believes this causes too much pressure for students, and would rather look at accuracy of the words read, without focusing on the rate of reading. She is comfortable with using a cutoff time for each word attempt to aid in the measurement process. The goal for the class is for students to be able to read with a 94% accuracy rate at the “F” or “10” DRA level. She states that she does not mind if students do this slowly, as long as the accuracy rate is there. The target date for this goal is the first week of April.

Task Analysis

Mrs. D. uses the DRA as part of her school’s reading assessment policies and to help in choosing appropriate reading materials. The school also uses DIBELS three times per year in a school wide assessment. As for instruction, Mrs. D. draws from a vast variety of methods. She uses both DRA rated texts and Scholastic published Literacy Place basal readers as sources for reading materials. In the classroom, she teaches reading concepts through songs, clapping, writing on the chalkboard, worksheets, games, and other methods. She has no single stepped or graded curriculum which she follows, but rather draws from several sources in creating her own stepped curriculum.

The Ohio Department of Education (2004) has listed the following standards relevant to reading accuracy, divided into pre-kindergarten, kindergarten and first grade:

Pre-Kindergarten Skills:

1. Identify matching sounds and rhymes in familiar story or poem passages and songs.
2. Identify sounds in words by isolating their syllables using snapping, clapping or rhythmic movement (e.g., cat, cher-ry).
3. Be able to distinguish sounds that are different (e.g., environmental sounds, animal sounds, phonemes).
4. Recognize words which share phonemes and repeat the common phoneme (e.g., /b/ as in Bob, ball, baby; /t/ as in Matt, kite, boat).
5. Identify and list some upper and lower case letters in addition to those in the child’s first name.
6. Understand that words are made up of letters.

Kindergarten Skills:

1. Identify and complete rhyming words and patterns.
2. Count the number of syllables in words by using clapping, snapping or counting.
3. Recognize and name all upper- and lower-case letters.
4. Recognize, say and write the common sounds of letters.
5. Recognize that words are separated by spaces; individual letters are not.
6. Hear and repeat the separate phonemes in words; identify the initial consonant sound in a word and blend phonemes to say words.

Grade 1 Skills:

1. Identify and differentiate among letters, words and sentences.
2. Identify and repeat the beginning and ending sounds in words.
3. Demonstrate an understanding of letter-sound correspondence by saying the sounds from all letters and from a variety of letter patterns, such as consonant blends and long-and short-vowel patterns, and by matching sounds with their corresponding letters.
4. Decode using letter-sound matches.
5. Use knowledge of common word families (e.g., -ite or -ate) to sound out new words.
6. Blend up to four phonemes into words.
7. Add, delete or replace sounds in a word to create new or rhyming words.

The preceding hierarchies of skills are those which would be needed to meet the goal of Mrs. D.’s class: reading passages from the "F" or "10" level of the DRA with a minimum 94% accuracy.

CBM Creation

Mrs. D. states that many of her students entered first grade with little word or reading knowledge, and that some were not able to recognize several letters and did not know many individual letter sounds. Typical students are expected to read at the DRA "16" level by the end of the first grade (Beaver, 2001). However, typical students are much further along when entering first grade, not only knowing all letters and many sounds, but are fluent in the more basic of reading skills. Books for typical students for the second month of first grade level (DRA levels 3-6) contain 46 to 75 words. Students are expected to read these materials with 94% accuracy. Clearly, Mrs. D.’s students did not start at that level. Mrs. D. will also have approximately two months after this case study ends to continue progression towards end of year ambitions, but she does believe she is being optimistic to expect most of her students to be slightly below the standard first grade DRA expectations at the end of their first grade year.

DRA level "F" or "10" reading includes stories about children and problems children will be familiar with (Beaver, 2001). The passages include repetition of text (ex. "Monkey see. Monkey do. Monkey me. Monkey you."). An entire story from levels "8" to "14" contains between 87 and 207 words. The DRA system incorporates the concept of most children’s books of integrating text and pictures, but passages taken from DRA graded texts to be used for assessment in this case study will be individually typed and will not include illustrations.

Considering Mrs. D’s goals, CBMs were created to obtain a baseline score for the class’ performance. Each of the CBM style passages were approximately 50 words in length. Though higher performing students would have little difficulty with this short of a passage, the teacher believed that lower functioning students would be pushed to their frustration level if the passages were longer. Since most of the students in this class had already been identified as lower performing, this was considered the best level for all. DRA rated texts are leveled in part due to the amount of repetition of words within a passage (Beaver, 2001). For this reason, it was decided that for each passage, the total number of unique words would be divided by the total number of unique words read correctly to obtain a percentage accuracy score. An example of the passages is as follows:

I had a lot of messages today. My mom sent me a fax. It was a funny fax. My friend Samina called me on the phone. It was a long phone call. My granddad sent me a letter. It was a great letter. My cousin sent me an email on the internet.

The above is a 52 word passage containing 31 different words. For the above passage, if the child were to read 20 unique words correctly, the child’s accuracy score would be 20/31, or 64.5%.

Results and Decisions

CBM Analysis

The probes were administered three times for week one, and the median score was taken for each child. If a child was not available for three probes, the average of two probes was taken. The probe was again given one time to each child for a second and third week to show a class wide trend. Probes were administered in the morning school hours on Wednesdays, Thursdays, and Fridays. Students were tested one at a time outside the classroom, at a desk in the hallway. The hallway was busy with students and other teams of students working nearby. There was never a time when the hallway was quiet or empty. The students in this school seemed accustomed to working in the midst of others.

The results of the baseline CBM data are mixed; however, discussion with the classroom teacher confirms that the results appeared both typical of the class. Teacher reporting shows that this class is more variable in their progress than many other classes she has had. She commented that students appear to do well some weeks, and regress during others. If one were to look solely at the class median scores, a trend of doing better on the second week’s probe followed by a regression to below first week levels on the third week’s probe would be seen. For the three weeks, the median accuracy percent scores were, 70, 72, and 66.6 respectively. This pattern can also be seen for the first quartile scores, but with even greater variability. However, for the 75th percentile scores, there was no improvement the second week; all subsequent accuracy scores after the first week regressed (87.7%, 84%, and 77.9%, chronologically). Due to the limitations of this study, it will remain unknown if the scores would have been the same if the median of three probes would have been used for the second and third weeks.

Chosen Subject

The subject of the study will be referred to as Justin. For all three weeks, Justin scored below the 25th percentile of the class. For week 1, Justin’s percentile was 4.2, with an accuracy score of 38.4%. For week two, Justin scored at the 15.8 percentile with an accuracy rate of 64%, and for week three, his percentile was 4.2 with an accuracy rate of 42.4%. Justin was the lowest scorer in the class for the first and third week of baseline testing, and was the third lowest the second week of testing. Justin’s percent accuracy scores were more variable across the three weeks than any of his fellow students. The teacher reports that his variability is also typical of his past performance.

Trend lines for Justin and the class show that the skills of the two are variable. Were simple probes and statistics 100% accurate, the skills of the two would meet in approximately five weeks. Because of this, goals for Justin will not be built on having him progress at the same rate as the class. Instead, goals for Justin will be based on the average of the three 25th percentile scores for the baseline period. It is the combined goal of the teacher and myself to see Justin improve to constant minimum accuracy rate of 61.3% by April, 2004. Justin’s current three week average accuracy rate is 48.2%, but it should be noted that the teacher felt his second week score (64%) was abnormally high. This goal would show an increase of 13.1% in accuracy. As is the nature of students, these goals may be adjusted or improved if needed.

References

Beaver, J. (2001) Developmental reading assessment: K-3 Teacher resource guide (Rev. ed.).Parsippany, NJ: Celebration Press.

Ohio Department of Education (January 27, 2004). Academic Content Standards. Retrieved January 27, 2003, from http://ims.ode.state.oh.us/ODE/IMS/ACS/Default.asp.

Filed under: EDC 514-515 Academic Assessment for Intervention
Copyright: February, 2004 - David Profitt