• Appendix E and F from the Essentials of KABC-II (2004) book. Appendix E represents socioeconomic status norms that convert the KABC-II global scores of children ages 3-6 years to a percentile rank based on their mother's level of education. Appendix F provides the same information only for children ages 7 to 18 years.

Appendix E

Appendix F

• Following is an interview with Dr. Mark Daniel and Dr. Elaine Fletcher-Janzen about a study conducted in 2004-2005 that investigated the relationship of socioeconomic status to scores on the WISC-IV and KABC-II with diverse groups of students.

What made you want to undertake this study?

The original KABC had a long history of smaller ethnic global score differences and a history of Alan and Nadeen Kaufman trying to accommodate for the influence of socioeconomic variables on scores. That is why they developed the sociocultural norms that were placed in the original manual for the KABC. So it was a natural extension of the investigation of ecological validity for the new KABC-II.

There were some questions that we had about the new KABC-II that needed to be investigated, especially questions about how scores are affected by culture. The questions that we were concerned with were questions like “How does ethnic group membership affect scores?”; “How does poverty affect scores?”; “Are some group scores affected more than others?”; and “Does level of acculturation affect group scores?”

We learned from the Taos Pueblo Study that ethnographic information is critical to the study of ecological validity of a test battery. In that study, the ethnographic information gave meaning to the scores. Without it, we would have made radically different decisions about what the scores meant for the Taos children. The Taos study experience brought more questions, as most studies do, about other ethnic groups. Would Hmong children have a different profile than African American children?Would different groups of Spanish bilingual individuals address the KABC-II in a different manner?

All sorts of questions come up when you have a tool that was designed to reduce cultural differences in scores, and frankly, these investigations are really fascinating! We have learned a lot from using anthropological techniques, and in a way, we were forced to use these techniques because ethnicity is a very nebulous construct, and it intersects with many variables that affect test scores. The examiners for the different ethnic groups were instructed to collect as much ethnographic data and report that with the quantitative data. At this time, we are still trying to sort out the relationship of each ethnic group’s scores to the ethnographic information, and this will take some time. Of course, we were diverted from the ethnography because the effects of SES were so powerful, and we are trying to put that information in context so it can be helpful to examiners who use the KABC-II and even the KTEA-II (Kaufman Test of Educational Achievement, Second Edition).

What were your initial hypotheses? Were there specific aspects of the research that you wanted to look at?

We really did not have an initial hypothesis per se. We wanted to conduct an in-depth study of homogenous ethnic groups and investigate the quantitative and qualitative aspects of their performance on the KABC-II and the WISC-IV. We wanted the ecological aspects of our study to lead the way, so to speak. Of course, we had some expectations that were based on our knowledge of the original KABC and also the standardization data of the KABC-II. We expected, for example, that the different ethnic groups would have slightly higher MPI than FCI indexes and that most of the groups would have scores on the KABC-II very close to the norm sample mean. When the scores came in for the different groups, however, our basic expectations were not met, and that is where the study became very interesting indeed!

How did you design the study? Were there any unique challenges in setting it up?

[Dr. Mark H. Daniel, Director of Psychometric Development at Pearson Assessments, contributed the following insights.]

The study was set up toward the end of the standardization period for the KABC-II, which had several benefits. The site coordinators and examiners who were invited to participate in the study were very familiar with the KABC-II standardization edition and had shown mastery of the administration and scoring procedures. This was important because there is a learning curve with any new assessment instrument, which could have compromised the validity of the comparison with WISC-IV results. Each participant selected one or two distinct and relatively homogeneous cultural groups to focus on, ensuring that the psychologists felt comfortable working with those groups. In addition, the participants had an interest in the general topic of the effects of culture on cognitive assessment.

The publisher of the KABC-II provided all necessary test materials and compensated both the examiners and the children and adolescents for their time but imposed few restrictions on how each study was conducted. The administration order of the two instruments had to be counterbalanced, and the time interval between administrations had to be within certain bounds. The number of examinees with special-education classifications or clinical diagnoses within each group was limited to no more than about 10%. Beyond that, each site coordinator decided on the selection of examinees and the test administration process. The publisher made it clear to the participants that the study must be conducted fairly, with no preconception of how the results would turn out.

Talk about your findings. How did they relate to your original hypotheses?
 
This study sought to evaluate two modern cognitive ability batteries based on the psychometric abilities model, the KABC-II and the WISC-IV, with respect to their sensitivity to SES in several ethnic groups that vary considerably in SES and acculturation. The six groups included in this study were African American (Virginia), Hispanic (Virginia and California), Hmong (California), American Indian (Lakota Sioux), and Native Hawaiian. A total of 174 children and adolescents (about 30 in each group) were administered the two instruments in counterbalanced order.

The first result of interest is average scores for each sample on the global score of each battery (the FSIQ of the WISC-IV, and the Fluid-Crystallized Index [FCI] of the KABC-II). The correlation between the FSIQ and FCI was .84 in the total sample of 174 children and adolescents. On both tests, the Native Hawaiian sample had the highest mean scores, closely followed by the African American sample; means for both groups fell between 95 and 100 on both instruments. The pattern of mean scores for the other four samples differed between the two instruments, being much more variable on the WISC-IV than on the KABC-II. Two groups, the Hispanic (California) and Hmong samples, scored substantially lower on the WISC-IV than on the KABC-II; the Hispanic (Virginia) sample showed similar scores on both tests; and the American Indian sample scored several points higher on the WISC-IV than on the KABC-II.

The next step in the analysis was to assess the influence of SES, indexed by mother’s level of education, on test scores and on the pattern of group mean scores. In the total sample of 174 examinees, SES level correlated .39 with the WISC-IV FSIQ and .25 with the KABC-II FCI; the difference between these correlations is statistically significant (p < .01). That is, SES explains about 15% of the variance of WISC-IV scores and about 6% of the variance of KABC-II scores. High-SES examinees in the study tended to score higher on the WISC-IV than on the KABC-II, whereas the reverse was true for low-SES examinees. It is noteworthy that the two groups showing substantially higher mean scores on the KABC-II than on the WISC-IV were the two samples with extremely low distributions of SES.

Why are these results important?

Since the beginning of mental ability testing, we have known that SES accounts for much of the variance in scores—Alfred Binet wrote about it in 1909! There are lots of reasons for this phenomenon; however, it was not necessarily clear that how you define intelligence or mental ability can change how a test is affected by SES. This study shows that the design of the KABCII, although highly correlated with the other popular measures of ability, is less affected by SES than are those other tests. The fact that children from low-SES backgrounds consistently score higher on the KABC-II should suggest to examiners that they must have assistance in making culturally competent choices when assessing children from low-SES backgrounds.

What are the practical implications for the field?

The practical implications are that examiners who work with low-SES children have a choice as to which instruments they use with this population. The question that is difficult for examiners when working with children from poor neighborhoods is "Does this score represent the child's current brain-behavioral functioning or does it reflect poverty?" Many low-SES children have fewer opportunities to learn, less emphasis on education in the home, poor nutrition, poor health care, and lots of other disadvantages that interfere with their ability to learn. The examiner has to pick apart what is ability and what is environment. That is why Alan and Nadeen published socioeconomic percentile score tables for low-SES children in the Essentials of the KABC-II book.

An examiner can take the regular scores from the KABC-II and then see how the child compares to other children from low-SES backgrounds. If a child has a lower score on one of the indexes that is troubling the examiner, he or she can go to the SES tables and see if the score is still low when it is compared to other children of low SES. If the new comparison score is still low, then the examiner knows that most children of low-SES backgrounds get this kind of score. Sometimes the comparison score is higher; then the examiner knows that the child in question is doing quite well, notwithstanding background factors. Comparing a child to other low-SES children does not provide a definitive answer, but it does give the examiner more information to try and understand ability versus environment.

Will you follow up this study? What aspects would you like to pursue?

Absolutely. We need to further understand the best ways to define socioeconomic status (in this study, mother's level of education was a proxy variable)—And if you change the definition, does the relationship to cognitive ability change? An implication of these findings is that studies of ethnic group differences in cognitive abilities may be affected by the degree to which the measuring instrument is influenced by SES or acculturation, even when considering instruments that are very similar in their theoretical underpinnings and their content and that intercorrelate highly. So examiners now have choices to make when working with children from low-SES backgrounds, many of whom come from ethnic minority groups. The effect of tests scores on eligibility for special education, for example, is an important area of research right now.

Therefore, an aspect of future research will center on overrepresentation of ethnic minorities with special-education labels such as mental retardation. We want to look at the practical implications of this study and whether these findings actually affect decision making in the public schools. After all, research should inform practice!


Copyright © 2006 Pearson Assessments