Psychoeducational Assessment - the final question.

David Profitt
Final Exam, EDC 512, Cognitive Assessment

Question

Describe the problems associated with traditional psychoeducational assessment practices. What is considered “best practice” in cognitive assessment?

Background

Though the focus in many school psychology educational systems is turning to modern and non-traditional assessment methods, the average school psychologist spends several hours per week administering and utilizing standardized academic and cognitive assessment tests (Reschly & Grimes, 2002). These tests are administered to children outside of the traditional classroom’s natural environment in often strange and unfamiliar settings. The students’ performance on a single set of tests is often used to judge the current and predicted abilities to perform in the classroom. These predictions are then used to place students in diagnostic learning categories such as Learning Disabled or Mentally Retarded. In many states, it is often placement in these categories which determines whether or not students qualify to receive assistance needed to alleviate educational problems.

Problems with Traditional Assessment Techniques

There are several problems with this traditional due process of assistance qualification. No matter how standardized the testing, there is no possibility of standardizing the rapport between student and tester, the current mood of the student, the student’s previous night’s sleep, and any of the other many factors which can greatly influence test performance. To base placement decisions on the performance of a single testing period is unfair to those children who could have done better given different circumstances, as well as unfair to those children who would normally have done worse.

Beyond the possible flaws in the testing session are the presumptions of accuracy, validity and usefulness of the test scores. Many of the popular intelligence tests give basic scores of full scale IQ divided into subscales such as performance IQ or verbal IQ (Reschly & Grimes, 2002). As with all scores, these scores contain error. This fact of testing must be accepted, as an errorless test for human cognitive or academic assessment has not and most likely will not be created. However, many popular intelligence tests, such as the Wechsler series of tests suggest calculating subtests profile analyses (Gresham & Witt, 1997 and Wechsler, 2002). In doing this, the tester is taking inherently erred scores and using them to calculate scores which will contain greater error as a result. These scores are then used in making placement decisions.

More often than solely using the scores from the administration of a single cognitive test, many school psychologists calculate the discrepancy between a cognitive or aptitude test score and a separate achievement or academic score. Many tests series, such as the Woodcock Johnson or the Wechsler series of tests offer both a cognitive and academic version of their tests for this purpose (J. Q. Morrison, personal communication, November 25, 2003). In following this method, school psychologists assume that a learning disability is defined as a minimum discrepancy between a student’s achievement score and aptitude score. The tests used to calculate these discrepancies generally contain a mean of 100 and a standard deviation of 15. A two standard deviation discrepancy is usually considered adequate to give the diagnoses of a learning disability.

This two standard deviation discrepancy may seem logical for those students who have a fairly high IQ, but obtaining a 30 point discrepancy score for an overall lower functioning student who is much more difficult. Consider this example. An extremely bright individual may have an overall IQ score of 120, but an academic math score of 85, giving a 35 point discrepancy. In many systems, this student would be classified as having a specific learning disability in math and would qualify for special services. However, if another student had an aptitude score of 90, which is still in the average range; that student would have to receive a math score of 60 to be considered to have a specific learning disability in math. It seems unfair that a low average student would be required to have such a low math score to receive special services. It is also less likely this student would score so poorly (nearly three standard deviations below the mean) on an achievement test in math. As can be seen in this example, students who are generally lower in abilities are often denied services, and slip through the cracks.

The discrepancy formula described assumes that there is an inherent correlation between one’s IQ score and one’s academic achievement. However, when considering reading ability, Aaron (1997) reports that the IQ – achievement correlation seldom exceeds .50. This means that no more than 25% of the variance in reading achievement scores can be attributed to IQ. If this is the case, one must ask the question of why we often use IQ scores in the formula which determines eligibility placement.

In California, the California State Department of Education has banned the use of Intelligence tests (Jacob & Hartshorne, 2003). This resulted from the lawsuit, Larry P. vs. Riles (1984) in which the court decided that the schools could not use IQ tests to assign African-American students to any special education program, with the exception of gifted programs. In 1992, the same judge, in response to parental suites, lifted the ban for cases in which the parents consented to having their children tested, but the state continues to prohibit the use of the tests. Attempts by the APA to challenge this ban have been unsuccessful.

Best Practices for Using Standardized Assessment Tests

Fortunately, the discrepancy formula has received a growing amount of criticism (Aaron, 1997) and in Ohio, has been replaced by the Specific Learning Disability Eligibility Tool Kit (Ohio Department of Education, n.d.). This new practice outlines the requirements for a data based assessment model of determining eligibility for specific learning disabilities.

If the academic placement and intervention profession is to discontinue use of the discrepancy model, what would be considered “best practice” in using academic and achievement tests in determining eligibility and intervention for students in need? According to the Reschly and Grimes (2002) article in Best Practices in School Psychology IV, best practices in cognitive assessment include the following:

• When using standardized tests, the goal should be for intervention, rather than simple eligibility placement. This brings to mind the thoughts of Messick (1995), who would warn against using standardized scores which may negatively label a child. We must look at the purpose of testing. If a test is used, it must be used for the sole purpose of providing assistance. One needs to look beyond the construct validity of a test to see the relevance or utility of the test to the situation, the value implications of using the test, and the social consequences which will be had if the test is used.
  
• If an intelligence or academic achievement test is to be used, it should be used with a purpose. Rather than taking stabs in the dark, looking for disabilities, the tests should be used in gathering information about a specific problem for which a child has been referred. If intelligent test results are not be relevant to the question at hand, the test should not be used.
  
• When an intelligence measure is used, it should be used in conjunction with other forms of assessment. The strengths and weaknesses a student shown in other areas, such as in classrooms and other various settings, must also be considered. Also, the choice of test should be for the benefit of the child and chosen with the abilities of the child in mind. Educational psychologists can not consider one or two tests in their arsenal to be adequate for the many different aptitude levels and unique characteristics of children. Test givers must keep current with the newest publications of tests, so they are not comparing students to outdated norms.
  
• Test scores are not absolute points. It must be remembered that scores should be reported to reflect the range of a child’s possible true score. Scores should be explained to parents and teachers in this fashion, and the less accurate scores (such as subtest profile analysis scores) should not be used at all. Continuing with the idea that scores are not absolutes, it must be noted in reports, and specifically pointed out to teachers and parents that IQ scores can change over time. IQ scores are a statement of current performance; caution should be used concerning their predictive value. Aaron (1997) speaks of the reciprocal nature of IQ tests and ability. In the realm of reading, it is true that IQ scores may relate to reading abilities, but continued practice in reading will also lead to greater general knowledge, which may increase IQ scores.

Alternative Assessment Methods

There are several methods of academic assessment outside of the testing arena. Direct assessment involves assessing academic performance directly (Gresham & Witt, 1997). Assessment procedures use absolute measures, such as words per minute for reading, or problems completed correctly for math. Readings can be taken in the classroom and comparisons to past direct assessment scores can be made to evaluate the student’s current performance and improvement. Comparison to classmate scores can be used to show a student’s knowledge relative to that of the class; comparisons can also be made on a school system, state, or national level. These scores are direct observations of how a student is performing. Specific difficulties with the required curriculum and individual weaknesses can be pinpointed, showing where instruction needs to be enforced.

Aaron (1997) also points out that assessment can be done with and without reinforcement. This will show whether the problem issue is a skills deficit or a performance deficit. This form of direct assessment not only points out the specific academic problem areas, but it also may give clues to the nature of the difficulty, and thus the assessment lends itself to intervention. If scores improve with reinforcers, environmental changes may encourage better performance. If scores remain constant regardless of reinforcers, then it is more likely that specific skills training is needed.

In utilizing direct assessment methods, the student does not need to qualify for assistance based on an abstract score. Additionally, the assessment procedures can be repeated to monitor progress. Testing can be done in the classroom, and in most cases, every student can participate. This monitoring will help prevent marginal students from being missed, as can be the case with standardized assessment.

Personal Thoughts/Future Concerns

If more and more schools change to a direct assessment model, it would be interesting to study the long term results. Will comparing students to local norms allow for a decrease in local performance over time? If class norms include the students who are struggling, the benchmark of acceptance may be lowered. Over time, this could make it appear that the lower functioning students are not so far below their counterparts, and the requirements of academic performance could be decreased for the group as a whole.

References

Aaron, P. G. (1997). The impending demise of the discrepancy formula. Review of Educational Research, 67 (4), 461-502.

Gresham, F. M., & Witt, J. C. (1997). Utility of intelligence tests for treatment planning, classification, and placement decisions: Recent empirical findings and future directions. School Psychology Quarterly, 12(3), 249-267.

Jacob, S., & Hartshorne, T. S. (2003). Ethics and Law for School Psychologists Fourth Edition, New Jersey: John Wiley & Sons, Inc.

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performance as scientific inquiry into score meaning. American Psychologist, 50, 741-749.

Ohio Department of Education (n.d.). Introduction to Specific Learning Disability Eligibility Tool Kit. [Brochure].

Reschly, D. J., & Grimes, J. P. (2002). Best practices in intellectual assessment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology (4th ed.) (pp. 1337-1350). Bethesda, MD: The National Association of School Psychologists.

Wechsler, D. (2002). WPPSI-III Administration and scoring manual, TX: The Psychological Corporation.

Filed under: EDC 512-513 Cognitive Assessment and Practicum
Copyright: December, 2003 - David Profitt