Small Business Resources, Business Advice and Forms from AllBusiness.com

The Race Gap in Student Achievement Scores: Longitudinal Evidence from a Racially Diverse School...

By Alvarez, R Michael
Publication: Policy Studies Journal
Date: Sunday, August 1 2004
HEADNOTE

Black and Hispanic students often display substantial gaps in test scores when compared to White students at all levels of education. In this article, we examine when and how the Black-White and Hispanic-White test score gaps

develop in the early elementary grades in a California school district with a large minority population, where more than 80% of the students are Black or Hispanic. We use multivariate analysis to predict the annual reading and math test scores of a student cohort from first through fourth grade controlling for various school and family factors. We find that in this racially diverse school district achievement gaps do develop, for both Black and Hispanic students. However, in comparison to the Black-White achievement gaps, the Hispanic-White gaps develop later, in particular in math, and they are half the size of the Black-White achievement gaps. The eventual widening of the gaps for Hispanic and Black students does not seem to he the result of minority students attending schools of less quality. Finally, in contrast to previous studies with fewer minorities, the estimated achievement gaps by the fourth grade are small.1

One of the most pressing concerns in American public education today is the so-called race gap in student achievement test scores. The "race gap/' usually studied as the difference between Black and White students' achievement scores, clearly and repeatedly arises across the nation. For example, in the 2003 California Stanford 9 student achievement data, 23% and 29% of Black second graders were assessed as proficient or better on the reading and mathematics tests respectively, relative to 50% and 61% of White second graders.2 Similarly, among California's Hispanic students, only 17% and 30% of second graders tested proficient on reading and mathematics.3

A widely ranging set of factors have been asserted to explain these racial differences in student achievement test scores, with most attention focused on students' background and family (e.g., Brooks-Gunn, Klebanov, & Duncan, 1996; Herrnstein & Murray, 1994; Phillips et al., 1998a), or school and educational policy issues (e.g., Cook & Evans, 2000; Ferguson, 1998). In general, this research has shown that even after careful and detailed controls for family and school backgrounds, a substantial and puzzling gap in achievement scores still persists, though more recent work, with a larger number of covariates, has found narrow or nonexistent gaps for incoming kindergarten students (Fryer & Levitt, 2003).

To further probe the causes for the race gaps, researchers have begun to examine when these gaps appear and how they develop. From this research we know that the Black-White achievement gaps arise sometime in the early grades and widen by the end of high school. However, research results in this area are often conflicting, possibly due to the fact that many studies use different measurement tools and span different time periods; further, such studies are in short supply, possibly due to the sensitivity of the subject. For example, national studies have shown varied results regarding when the Black-White achievement gaps precisely arise, with some studies showing an earlier onset of the gap, in kindergarten, and other studies showing a later onset, in early elementary grades (Fryer & Levitt, 2003; Phillips, 1998). Similarly, although we know Black students finish high school with larger test score gaps than when they entered first grade, studies on the evolution of these gaps throughout elementary and high school yield varied results (Phillips, Grouse, & Ralph, 1998).

Surely, more research on the onset and evolution of the gaps is needed. One way to contribute to the study of the dynamics of the race gaps is to examine data sets with larger samples of minority students (Phillips, 1998; Phillips, Grouse et al., 1998). Many national and state and district level studies have focused on samples with small percentages of minority students that may not significantly capture the educational experience of minority students nor sufficiently sample from their various backgrounds. For example, Hispanics and recent immigrants are often underrepresented in these data.4 In the present study, we have the unique opportunity to investigate when and how the Black-White and Hispanic-White gaps develop in the context of an urban school district in which Black and Hispanic students combined are more than 80% of the student population. Thus we have a large sample of minority students (including recent immigrants), and we can also study the evolution of the Black and Hispanic gaps in a racially diverse school environment.

The picture that emerges from our research suggests that, as in studies with a majority of White students, in a diverse school district achievement gaps do develop, both for Black and Hispanic students. However, when and how the gaps develop varies by racial group. In particular, we find that Black students have significant test score gaps with respect to White students in the first grade, whereas Hispanic students' gaps become significant in the second grade (especially in math). Moreover, as the gaps widen in later grades for both Black and Hispanic students, Hispanics' gaps are consistently smaller than Black students' gaps, often half the size. These results hold even after controlling for school quality and students' initial test scores. Overall, in contrast to previous national studies with less minority representation, our estimated achievement gaps by the fourth grade are substantively smaller.

From a theoretical and policy perspective, our findings have important implications. The differential onset, dynamic and sizes of the achievement gaps for Black and Hispanic students imply that the explanations and sought causes for the race gaps, in particular, family versus school factors, may differ for each race. This in turn implies that different policies might help eliminate them.

Previous Research on the Dynamics of the Race Gaps in Test Scores

Race gaps in test scores are undisputed facts. Looking at national reading scores shows that Black students at age 9 average close to 0.9 standard deviations below White students, whereas Hispanic students at this age score close to 0.72 standard deviations below White students (Nation's Report Card, 1999). Similar gaps are observed in math. As evidenced in many studies, when achievement test scores are examined after controlling for a large array of individual, family and school factors, substantial racial gaps still remain.5

Increasingly, scholars are studying when these gaps arise and how they evolve. These sorts of questions allow researchers to further explore the causes for the race gaps, by giving different importance to family versus school explanations depending on the size of the gaps in the early grades and their evolution (Phillips, 2000). For example, a later onset of the Hispanic-White gap relative to the Black-White gap may give more weight to school and language factors as explanations for Hispanics' gaps, whereas it may point to preschools, families, or neighborhood environments as explanations for Black students' gaps. However, the evidence in this research field is mixed, in particular, regarding the onset of the gaps.

According to several national longitudinal studies, the Black-White gap in reading scores starts in first grade or earlier, as seen in studies based on Prospects' data (Phillips, 1998,2000) whereas other analyses find a later onset, as seen in studies based on Children of the National Longitudinal Survey of Youth (CNLSY) or studies based on Beginning School Study (BSS) data (Phillips, 1998, 2000; Entwisle & Alexander, 1988). For Hispanics, many studies have shown an early onset of the gap, already by the first grade, both in reading and in math (Phillips 1998, 2000). In terms of the evolution of the gap, a meta-analysis of various national studies finds that the math gap widens by 0.18 standard deviations between first and twelfth grade, but the reading gap does not widen at all (Phillips, Grouse et al., 1998). In contrast, other national studies examining IQ test scores or other scholastic measures have found few age-related changes (Gordon, 1984; Hauser & Huang, 1996; Loehlin, Lindzey & Spuhler, 1975).

One way in which studies of the dynamics of racial gaps could be improved is by having sample designs that include more minorities, start in kindergarten, and follow students through high school, among others (Phillips, 2000). A recent study by Fryer and Levitt (2003) examines a large national cohort of students from fall of kindergarten through spring of first grade, using the new survey data from the Early Childhood Longitudinal Study (ECLS). Quite promisingly, their novel data set (which includes a large array of covariates) indicates that the race gap in kindergarten between Black and White students in math and reading essentially disappears once sufficient controls are introduced. The onset of the test scores gap for Black students seems to occur in first grade. Fryer and Levitt examine a variety of possible causes for the appearance of the Black-White race gap after kindergarten, and conjecture that Black students attending inferior schools in the primary school years can play an important role. However, their study only spans kindergarten through first grade, and more data and analysis of their cohort throughout the next years is needed.

As discussed in later sections of this article, we examine longitudinally the racial gaps for the first four grades in a school district in which Black and Hispanic students constitute more than 80% of the student body. The large presence of minority students, including recent immigrant Hispanic students, allows us to study a potentially wide sample of educational experiences and backgrounds.

Moreover, though our analysis may not easily generalize to other districts, one of the strengths of our approach is precisely the opportunity to assess the evidence on the dynamics of racial gaps in a diverse racial environment. Apart from family background and school quality explanations, researchers of the achievement gaps have examined the role of the schooling environment towards minority students, such as teachers' expectations and teachers' race and peer effects, when trying to explain differential educational outcomes across races (Ehrenberg, Goldhaber, & Brewer, 1995; Ferguson, 1991, 1998). This research has raised questions about whether the racial makeup of the school, both in terms of the student population and body of teachers and administrators, has an effect on student performance (Entwisle & Alexander, 1994; Ferguson, 1998). For example, some studies have found correlations between the racial makeup of the school and more encouraging attitudes towards academic success (Cook & Ludwig, 1998), or better educational outcomes and less discriminatory acts toward minorities when in the presence of a more racially representative teacher and administration body (Meier & Stewart, 1991; Meier, Wrinkle, & Polinard, 1999). Our database allows us to answer whether a school environment with a majority of Black and Hispanic students gives rise to achievement gaps, and to examine whether these gaps are any different, in terms of onset and evolution, with respect to those from national studies with fewer minorities in their samples.

Student Achievement in a Diverse School District

Our study uses a unique database, containing the reading and math test scores and a wealth of background and school environment information of each student attending the Pasadena Unified School District (PUSD) from 1999 through 2002. We show in Table 1 some comparative data for PUSD students in the 2001-2002 academic year, when over 23,000 students enrolled in the PUSD that year. As compared to the basic profile of California public school students, the PUSD is more racially diverse, with a higher proportion of Black and Hispanic students than the state's total. Socioeconomically, PUSD has a student body that is poorer and more disadvantaged than the statewide student body. Despite these demographics differences, the basic testing score data for the PUSD compares favorably to the state average, in particular in the early grades.

A distinct feature of the PUSD is its diverse racial composition. Previous studies, mainly of national scope, have often dealt with data sets with small samples of minority students. For example, the three most recent national longitudinal studies mentioned earlier had samples with 18% Black and 8% Hispanic students (CNLSY), 16% Black and 10% Hispanic students (Prospects), and 16% Black and 19% Hispanic students (ECLS). In contrast, the PUSD has 29% Black students and 52% of Hispanic students, thus providing an excellent opportunity to examine the dynamics of the racial gaps when minorities are a majority group.

IMAGE TABLE 1

Table 1. PUSD compared to California's public school students, 2001-2002

In addition to its large representation of minority students, the PUSD can be described as racially diverse from all students' perspectives. According to estimates by Frankenberg, Lee, and Orfield (2003) the average White public school student in the United States attends a school in which minority students make up 16% of the student body, however, in the PUSD, the average White student attends a school that is 66% minority. The average Hispanic public school student in the United States attends a school that is 65% minority (12% Black and 53% Hispanic), whereas the average Hispanic student in the PUSD attends a school that is 82% minority (24% Black and 58% Hispanic). Finally, the average Black student in a U.S. public school also attends a school that is 65% minority (though 54% Black, and 11% Hispanic), and the average Black student in the PUSD attends a school that is 82% minority (34% Black, and 48% Hispanic).

Despite the clear advantages from our district of study, we recognize that our analysis covers students from only one district, and as such, generalizations to other districts, and to other states, should be taken with some degree of caution.

The Data

The main PUSD student cohort that we focus our attention on consists of students who were in fourth grade in 2002, who had been in the PUSD since 1999 (first grade), and who had test scores for all four years.6 This cohort consists of 1,147 students for reading scores and 1,221 students for math scores, and they attended the 22 elementary schools within the PUSD. We refer to these students in general as fourth-grade 2002 cohort, and give a basic profile of the "reading" cohort (N = 1,147) in Table 2. The first column shows the summary for the entire group ("Full Sample"), and each racial group is profiled in subsequent columns.

The first row gives the mean reading Stanford 9 scores for each sample of students in the "reading" fourth-grade cohort in 2002. California students are mandated by law to take Stanford 9 tests in reading and math beginning in the second grade.7 This emphasis on standardization means we do not have in our data set other achievement metric than Stanford 9 tests. On the other hand, PUSD has required its first graders to also take the Stanford 9 tests, allowing us to assess the gaps early on with the same metric.

Further down the table, we present summary statistics for all the independent variables used in the multivariate analysis, classifying them into individual/family variables and school related variables. A voluminous literature exists on many of these variables and we will not discuss them in detail. For example, in terms of individual level variables that pertain to a student's background and family characteristics ample evidence has linked socioeconomic status, family structure, and language fluency to achievement outcomes.8 The remaining variables pertain to attributes of the school the student attends; these school attributes have also been studied thoroughly in previous research on student achievement though, in general, there is less consensus regarding the effect of school factors.9

In Table 2 we see that Hispanic and Black students are at least two times more likely to participate in the free lunch program than White students, and their estimated SES level is much lower than that of White students. Hispanic students are more likely to be in a school environment with a majority of Hispanic students, whereas Black students are more likely to be in a school with more Black students. White students on the other hand are more likely to be in schools with more fully credentialed teachers (though the differences are quite small) and are less likely to be in schools with higher concentrations of minority teachers. Each of these is a factor that could account for some of the race gaps in student achievement scores and is therefore controlled for in our multivariate analysis.

The Dynamics of the Race Gaps in the Early Grades

We begin our analyses with a cross tabulation of the dynamics of the race gaps in the early grades of the PUSD. We examine the mean reading and math scores from first grade through fourth grade for the fourth-grade 2002 cohort. The upper panel of Table 3 provides the raw testing data, and the lower panel computes the differences between Hispanic and White, and Black and White students. These should be considered as the basic racial gaps in student achievement in the early grades of the PUSD.

What we find is instructive. In the first grade, the average reading score of Hispanic students is more than 13 points lower than that of White students, and Black students' average is over 6 points lower than that of White students. By fourth grade, Hispanics' reading gaps are slightly reduced, by less than 1 point, whereas Black students' gap increased, by close to 2 points. The story is somewhat different in math. In the first grade, both Hispanic and Black students average in math around 11 points below White students. By fourth grade, Hispanics have reduced their gap in math, by close to 3 points, and Black students have slightly reduced it, by 1 point. Thus in reading the Black-White student gap increases, whereas the Hispanic-White gap slightly decreases; however, in math, we observe an overall decrease in both the Black-White and Hispanic-White gaps. Quite importantly, in the raw scores Hispanic-White gaps are in general larger than the Black-White gaps. These patterns in the raw data are quite similar to those found in some national studies (e.g., Prospects), but they differ from those found in other studies (e.g., CNLS). In general, the basic gaps in PUSD are smaller than those observed in longitudinal national studies, though we should keep in mind that different tests are being used in each case.

IMAGE TABLE 2

Table 2. Summary statistics by race for for PUSD students in fourth grade in 2002

IMAGE TABLE 3

Table 3. Achievement gaps in reading and math for fourth grade 2002 PUSD cohort

Multivariate Analyses and Estimated Racial Gaps

Our goal is to examine the dynamics of the racial gaps between first grade through fourth grade after controlling for student background and school attributes. To do so, we predicted students' test scores in a multivariate analysis using hierarchical linear models (HLM), which are linear regression models that more explicitly allow for a hierarchical structure in the data.10 In our study, we estimated three-level hierarchical linear models where test scores of a student, the 1st level, were "nested" in a student, the 2nd level, who was then "nested" in a school, the 3rd level. Equation (1) below presents the model estimated predicting test scores T, for achievement in grade g of student i in school s. The X's include the set of independent variables summarized in Table 2. The B's are the coefficients which can vary by grade; and the remaining term comprises the errors that in a hierarchical model can occur at different levels of the hierarchy.11

IMAGE FORMULA 4

We discuss first the results from predicting reading test scores, which are presented in Table 4. The first columns show the effects of the independent variables in the first grade, presenting the coefficients, their standard errors and their p-values. The subsequent columns show in a similar fashion the effects of the independent variables in the second, third and fourth grades. At the bottom of Table 4, the standard deviations of the errors are shown. Most of the variation is at the student level, whereas very little, around 3 points, is at the school level. The latter is not too surprising, given that the model is already accounting for many school level characteristics. The fit of the model is comparable to those obtained in other recent studies.12

The important coefficients for our discussion are in the first rows in bold type, and they express the estimated difference between Hispanic and Black students' reading scores and White students' scores, or the racial gaps. We see that in general both Hispanic and Black students in this cohort have lower reading scores than White students, controlling for the student and school attributes in our model. In the first grade, Hispanic students score close to 3 points below White students (or 0.15 of a standard deviation), and Black students score close to 4 points below White students (or 0.21 of a standard deviation).13 However, Hispanic students' estimated gap is just statistically significant at the 90% level, and similarly so, in the third grade. By fourth grade, Hispanic students again have reading scores around 3 points below White students, and Black students have increased their gap in reading to over 6 points (or 0.34 of a standard deviation). Both Hispanic and Black students experience an increase in the gap in the second grade; however, Hispanic students' gap narrows again in subsequent grades, but this does not occur for Black students.

The dynamics of the reading gaps are thus different for Hispanic and Black students in our PUSD sample: Hispanic students' gaps become statistically more established after the first grade (though between first and fourth grade they barely change in magnitude); and Black students' gaps are statistically significant at a high level of confidence throughout, and they increase over the same period of time. By the fourth grade, the Black-White gap in reading is twice as large as the Hispanic-White gap.14

Next, Table 5 reports the results for math test scores using the same methodology and model specification as in Table 4. The fit of the model is again comparable to those of other studies.15 For first grade students, we see a substantial Black-White gap, with Black students scoring more than 5 points below White students (or 0.28 of a standard deviation). Hispanic students, on the other hand, show a negligible gap with respect to White students (0.03 of a standard deviation), which is not statistically significant. In the second grade, the gap widens for Hispanic and Black students, becoming statistically significant for Hispanic students, and by fourth grade, both sets of students have increased their gaps relative to White students: Hispanics are scoring more than 2 points below White students (or 0.13 of a standard deviation, though p-value 0.12), and Black students are scoring 7 points below White students (or 0.35 of a standard deviation).

IMAGE TABLE 5

Table 4. Longitudinal Estimated Achievement Gaps in Reading Scores for Fourth Grade 2002 PUSD Cohort

IMAGE TABLE 6

Table 5. Lognitudianal Estimated Achievement Gaps in Math Scores for Fourth Grade 2002 PUSD Cohort

IMAGE TABLE 7

Table 6. Estimated Achievement Gaps Cross Sectionally for 2002 PUSD Students

Thus in the PUSD, the onset of the Hispanic-White gap in math is after the first grade, the onset of the Black-White gap is at or before the first grade, and both Black and Hispanic students increase their gaps in math in the early grades. By the fourth grade, Black students' gap in math is over twice the size of Hispanic students' gap.

Our analysis of the PUSD data provides evidence for three conclusions regarding the onset and evolution of the achievement gaps. First, after controlling for student and school attributes, the onset of the gaps for Hispanic and Black students differ, with Hispanics' developing after first grade, in particular in math. Second, the evolution of the race gaps varies by race and by subject: this is illustrated by the relatively unchanging gap in reading for Hispanic students and the comparatively expanding gap for Black students. And third, by fourth grade both Hispanic and Black students are scoring below White students, with Hispanic students' gaps being close to half of Black students' gaps in reading and in math. Overall, these results provide us with some insights into the potential causes for the onset of the gaps. In particular, the later onset of the achievement gaps for Hispanics, after the first grade and more sharply in math, suggests that school factors, and the interaction of language and school factors, may play a stronger role in the development of the Hispanic-White gap. In contrast, the earlier onset of the Black-White gap may suggest a stronger role for families and preschools.

The coming years will provide us with more data from this PUSD cohort to analyze how students' test scores evolve in later elementary years. In the meantime, because we have data for all students enrolled in the PUSD in 2002, we also examined cross-sectionally the achievement gaps from fourth grade through sixth grade to gain understanding of how the gaps may evolve. The results of this analysis are shown in Table 6; as this analysis is cross-sectional by grade, the set of students is not the same as in our analyses in Tables 4 and 5, and the sample varies for each grade.

We find that the Hispanic-White gap in reading slightly increases from 3 points to over 4 points between fourth and sixth grade (up to 0.22 of a standard deviation), whereas the Black-White gap substantially increases from 6 points in fourth grade to close to 10 points in sixth grade (up to 0.51 of a standard deviation). A similar pattern is found in math: the Hispanic-White gap does not change much, remaining at around 3 points between fourth and sixth grade (0.13 of a standard deviation), whereas the Black-White gap increases from 7 points to over 9 points (up to 0.46 of a standard deviation). These results, although cross-sectional, are quite suggestive of a continuing differential dynamic of the achievement gaps of Black and Hispanic students, with Black students' gaps continuing to increase at a higher rate than Hispanics' throughout the later elementary grades.

Explaining the Growing Gaps

We have found that the racial gaps develop and widen at different rates in the early grades for Hispanic and Black students in the PUSD. There are various explanations regarding why these gaps grow, as opposed to their onset, and they broadly fall into three categories: the enrollment of minorities in lower-quality schools, the inconsistencies of test measures across time that can potentially favor White students, and the changing importance of parenting versus school factors (Fryer & Levitt, 2003; Phillips, Grouse et al., 1998). We next examine these three accounts for the widening gaps.

The explanation of growing achievement gaps due to minorities attending schools of lesser quality does not find much support in our data. Our analysis includes three school attributes that account for or are proxies of many aspects of school quality. As shown in Tables 4 and 5, the gaps persist even after controlling for school attributes. To further investigate this explanation we considered alternative specifications: in particular, we reestimated our models with other school attributes (e.g., teachers' average years of experience), without school attributes but instead with fixed effects for the schools, and with a state level measure of a school's quality called the Academic Performance Index (API) based on students' test score performances.16 Table A.I presents the results for reading where we predict second through fourth grade scores controlling for a school's 1999 API index. In all cases, we find that the achievement gaps still remain even after these various school quality controls. Overall, we conclude that there is little evidence supporting school quality explanations for the growing gaps in the PUSD.

Next, regarding explanations of the widening of the achievement gaps based on tests examining different sets of skills as children age, potentially favoring White students, we argue that the Stanford 9 tests are constructed to be comparable in their outcomes across grades. That is, they are meant to target possibly differing though age-appropriate skills, including those from the early grades. Nevertheless, the problem of varying levels of difficulties is lessened once we examine the data from another vantage point: do the gaps still develop when we consider students with the same initial skills? We reestimated our main models by predicting tests scores from second through fourth grade controlling for scores from the first grade, as presented in Table A.2. We find that controlling for early skills, the Hispanic-White gap in reading develops by the fourth grade to 0.08 of a standard deviation (p-value of 0.24), and the Black-White gap in reading develops to 0.24 of a standard deviation (p-value 0.001). The estimates are consistent with our earlier observation of gaps developing in the early grades and evolving at different rates for Hispanic and Black students. The widening of the gaps therefore does not seem to be a by-product of standardized testing.17

Finally, we direct our attention to the remaining explanation for the widening gaps, the changing importance of parental and school factors with time and by race. For example, many Black students come from more disadvantaged background environments than their White peers. If family factors are more important for Black students than White students as they age, then Black students will eventually lag behind White students. In the case of Hispanic students, many of them face language acquisition and cultural insertion barriers (Fry, 2003; National Center for Education Statistics, 1995; Wojtkiewicz & Donate, 1995). If school and neighborhood factors are more important for Hispanic students than for White students, then Hispanic students will lag behind White students over time.

To address the differential effects of a factor by race and with time we estimated separate hierarchical regression models for Hispanic, Black and White students. That is, we replicated the analyses in Tables 4 and 5 but this time by race.18 Table 7 presents the results. For space considerations we only include the results for first and fourth grades.

These analyses provide several suggestive trends. In particular, the effect of family background as measured by enrollment in free lunch programs continues to be very strong for Black students, lowering their scores by over 10 points as they traverse the early grades. By comparison, the effect of free lunch enrollment becomes negligible for Hispanic and White students. By the fourth grade, having both parents in the household continues to have a positive effect for Black students, but for Hispanic and White students the effect is minimal. In fact, family structure dramatically drops in importance across grades for Hispanic students, whereas SES (which is related to a students' residential address, and may be capturing neighborhood effects) increases in importance. Overall, these results suggest that as children age family factors may play a diminishing role for White and Hispanic students but a continually strong role for Black students. Neighborhood effects may play a stronger role for Hispanic students as they age.

School factors, as one might expect, become more relevant as children age. In particular, by the fourth grade we find that more school factors become statistically significant for all sets of students and these in turn have policy implications. For instance, in the fourth grade higher rates of minority teachers differentially benefit Black students, and smaller class sizes differentially benefit Hispanic students.

A more definitive answer in terms of the changing role of parental and school factors by race requires larger samples, more detailed survey data, and is beyond the scope of this article. Nevertheless, our analyses provide much needed guidance into what may explain the evolving gaps, which in turn may help ameliorate these gaps. All in all, we have found that lower-quality schools for minorities and the inconsistencies of tests are not likely candidates to explain the widening gaps. We have found plenty of evidence that family and schools have a changing role in time, as one might expect, but more importantly, by race as well. Parental and environmental contributions may still disadvantage minority students in comparison to White students as they age. However, we have also found some school attributes that differentially benefit minority students.

IMAGE TABLE 8

Table 7. Estimated Achievement Gaps in Reading Scores for Fourth Grade 2002 PUSD Cohort by Race

How Do Our Results Compare to Those of Other Studies?

In the first sections, we noted other previous longitudinal studies that had examined the dynamics of the racial gaps in national samples. How do our results compare to these studies? Table 8 presents a summary of the results from our study compared to selected national longitudinal studies.19 The first column shows the achievement gap in the first grade expressed in standard deviations, the second column shows the linear yearly growth, or the average growth between first grade and fourth grade, if a linear growth model was not estimated (as in our study), and the last column shows the estimated gaps at fourth grade.

In terms of the initial gaps, or the gaps in the first grade, our results for Black and Hispanic students are smaller or comparable to the first grade gap estimates from Prospects and CNLSY data.20 Our racial gap estimates in math are smaller or comparable to those based on the ECLS data, though they are larger in reading. Fryer and Levitt (2003) estimated very small or no gaps in reading by the spring of the first grade: a -0.07 standard deviation for Blacks and a 0.001 for Hispanics. These gaps were among the smallest ever estimated. Our study finds gaps in reading of -0.21 and -0.15 standard deviations for Black and Hispanic students respectively, which are relatively quite small when we compare them to the first-grade estimates from the CNLSY and Prospects data.

By the fourth grade, our estimated gaps are always smaller than those obtained with the CNLSY and Prospects data-the ECLS data only goes up to first grade. For example, in the fourth grade our estimated gap for Black students in reading is -0.34 of a standard deviation while those of the other studies are above -0.7. For Hispanics, our estimated gap in reading is -0.15 of a standard deviation while the other studies obtain estimates of -0.23 and -0.67. We observe a similar pattern for math, with the estimated gaps from our sample being smaller than those from the other studies.

In general, our estimates are, both in the first and fourth grades, toward the lower end of the estimates found in the literature. Of course, direct comparisons between the results of these studies are not entirely appropriate because the tests used and the model specification vary from study to study. However, we can cautiously observe that with more controls, in particular parental and family background controls, and with a larger sample of minorities the gaps are substantively smaller than found in most previous studies.

Discussion and Conclusions

In this article, we have attempted to answer three questions. Do achievement gaps develop in a diverse racial environment? If these gaps develop, when and how do they develop? Last, how do the estimated gaps from the PUSD compare to those of other studies?

IMAGE TABLE 9

Table 8. Comparison of the Evolution of the Achievement Gaps in Standard Deviations Across Selected Studies

Our answer to the first question is a strong "yes": our data show gaps both for Hispanics and Blacks students in reading and math in the early grades. Our answer to the second question is "it depends on the racial group": Hispanic students show a slightly later onset of their gaps, in particular in math, than Black students; Black students experience a larger widening of their gap in reading than Hispanic students; and Hispanic students have in general smaller gaps than Black students. Our answer to the third question is a more qualified "they are on the small side": the estimated gaps in our sample tend to fall within the lower end of the estimated gaps found in the literature.

Our findings provide evidence that Hispanic and Black students experience different evolutions in their achievement gaps. This is in line with some previous studies in the literature, though in general the details of the dynamics vary greatly across studies (Phillips, 1998, 2000). In the PUSD, we find that Hispanic students experience a later onset of their math achievement gaps, more specifically after the first grade, than Black students. The potential causes for the onset of the Hispanic-White gaps, as opposed to their evolution, may lie more heavily on school, neighborhood and language factors.

The results imply that policy interventions that attempt to bridge the achievement gaps for Hispanic and Black students, of which there are many, may have to address the fact that slightly different factors may influencing them (for a recent review of promising polices, see Chubb & Loveless, 2002). For Hispanics, it may be the case that difficulties related to language or cultural differences play a strong role in particular in the very early grades. As suggested by some of our findings, smaller class sizes may differentially benefit them. Both for Black and Hispanic students, it may be that parental and more generally environmental effects play a stronger role as students age, in comparison to White students. Successful policies may have to address some of these differences.21

The fact that achievement gaps develop in the PUSD further confirms their pervasiveness: the achievement gaps for Hispanic and Black students also develop in racially diverse environments. However, quite intriguingly, our estimates for the test score gaps of Hispanic and Black students tend to fall toward the lower end of those previously found in published studies. On one hand, as we are relying on data from this single school district, perhaps the PUSD itself is unique and for some reason or combination of reasons the racial gaps and their dynamics differ substantially from those in other districts or states. But on the other hand, it is possible that the diversity of the student body in this district plays a role in diminishing the racial gap in student achievement scores. This latter possibility requires further research.

The fact that studies such as ours in the early grades, though only for one district, and that of Fryer and Levitt (2003) for the first grade at the national level, are finding relatively small gaps is quite promising in terms of our understanding of the race gaps. However, even if estimated gaps vanish, the "raw gaps" still remain and a lot of work still needs to be done to help break the linkages between race and socioeconomic and background conditions that are linked to lower scores.

FOOTNOTE

Notes

We thank William Bibbiani, formerly of the Research, Evaluation and Testing Department at Pasadena Unified School District, for providing the data (proprietary of PUSD) that we use in this article. An earlier version was presented at the 2002 Annual Meeting of the Midwest Political Science Association, and we thank Mark Schneider, Kenneth Meier, and David E. Campbell, among others, for their comments.

1. The data used in the analysis is proprietary to the Pasadena Unified School District, California. Data and coding material will be provided upon approval from PUSD.

2. see the website of California's Department of Education at http://www.cde.ca.gov for aggregate information on test results.

3. In this article, we focus only on the relative differences between Black, Hispanic and White students' test scores. Asian students are few in our sample (less than 2%). The dynamics of how Asian students tend to outperform White students is a worthy subject beyond the scope of this article.

4. Data from the Children of the National Longitudinal Survey of Youth (CNLSY), a much analyzed longitudinal study, includes only Mexican American children whose mothers have resided in the United States since 1979, potentially limiting the representation of recent immigrants. This set of students comprises only 8% of the sample.

5. For a broad survey of the Black-White gap, see Jencks and Phillips (1998), whereas for initial work on achievement gaps, see Coleman et al. (1966). Overall conditions of Hispanic students are described in National Center of Education Statistics (1995, 1998).

6. For example, students who are missing test scores in any of the 4 years or are missing SES values are dropped from the sample. We discuss the problem of panel attrition in the Appendix.

7. The test scores are normed curve equivalent (NCE) scores. They are obtained by first scaling the scores according to the difficulty of the questions. Next, these scaled scores are translated into a national percentile rank (NPR), which is the percentage of the national norming sample, with majority of White students that scored equal to or less than the student. Finally, the NPR is reexpressed as a value from (1 to 99) in a normal curve with mean 50.

8. For socioeconomic and family factors, see Brooks-Gunn, Duncan, & Klebanov (1994) and BrooksGunn, Klebanov, & Duncan (1996); McLanahan, and Sandefur (1994); and Phillips, Brooks-Gunn et al. (1998); and for work on bilingual education and English learners, see Rossell and Baker (1996).

9. For various discussions on school factors, see Currie and Thomas (1995); Darling-Hammond (2000); Ehrenberg et al. (1995); Ferguson (1998); Hanushek (1986,1999); Hedges and Greenwald (1994); Levin (1996); and Murnane (1975).

10. Hierarchical linear models can be viewed as linear regression models where intercepts and coefficients are allowed to vary systematically and randomly across sublevels of the hierarchy. In our study, we allowed for random individual and school intercepts. For a detailed discussion of HLM, see Raudenbush and Bryk (2002). All estimations were performed using HLM 5.05 (Scientific Software International). We also estimated our basic specification using other regression models, including linear regression and seemingly unrelated regressions. These alternative estimation approaches did not produce results that are substantively different from those in this article. We present the HLM specification because it is the appropriate specification for our problem and it encompasses simpler linear models.

11. More specifically, the error term has the following components: e^sub gis^, the error for a test in grade g for student i in school s; r^sub is^, the error of student i, in general; r^sub g=1is^, the error of student i in grade 1; and U^sub gs^ the error associated with school s in grade g.

12. For example, if we estimate the reading model using a simple linear regression model (no hierarchies in the errors), we obtain an R^sup 2^ of 0.31. In Fryer and Levitt's (2003) recent comprehensive study with a very large set of covariates, their R^sup 2^ for reading never surpasses 0.3. Studies based on the CNLSY data find adjusted R^sup 2^S no larger than 0.4 in reading for early grade students (see Jencks & Phillips, 1998).

13. The model in Table 4 predicts NCE test scores that are in a scale of 1 to 99. In the text we discuss gaps occurred in terms of NCE points but also in terms of standard deviations. The latter are obtained by dividing the coefficients for each race by the standard deviation of the raw scores for the whole sample of students, which is 19 for reading and 20 for math.

14. Although not the main focus of this study, examining the effects of some of the other independent variables across time illuminates the changing role of family and school factors. For example, in terms of background factors, family structure decreases in importance with time but SES increases with time. In the next section, we pick up some of these issues in relation to changes in the effect of these variables with time by race.

15. If we estimate a simple linear regression model (no hierarchies in the errors) then the R^sup 2^ is 0.2. For the Fryer and Levitt study (2003) with a larger set of covariates, though only for first grade the R^sup 2^ reaches up to 0.35.

16. We used API scores from the California Department of Education (see http://www. cde.ca.gov/pssa/api).

17. A lagged model with test scores from the previous year has its own set of methodological difficulties (e.g., multicollinearity) while also addressing a partly different question (i.e., the difference in scores between years). Because of this, we consider this alternative specification only as a test of robustness.

18. Previous empirical work provides evidence that family and parental factors can vary with time and they can vary by racial group, though their joint variation is often not examined. Our analyses in Tables 4 and 5 provide evidence of family factors decreasing with time (see also Fryer & Levitt, 2003). For example, having both parents in the home dramatically decreases in importance across grades. SES, which is a measure of the average residential value of where a student lives, and is therefore also measuring neighborhood effects, increases in importance with time. On the other hand, prior research has also found variations by race in terms of family and school factors (Bali & Alvarez, 2003; Fryer & Levitt, 2003; Jeynes, 2003). For example, cross-sectional work on the PUSD found that having both parents in the family had a stronger positive effect on minorities than White students, whereas enrollment in free lunch programs was associated with a stronger negative effect for Black students than Hispanic or White students.

19. We have summarized the results from these studies in a way to make comparisons more comparable across them. Any error in reformatting or reinterpretations of the results from other studies are entirely our own.

20. As seen in Table 8, the initial estimates for the CNLSY and Prospects data are for fall of first grade, whereas ours and those from the ECLS study are for spring of first grade. Using the estimates of the linear growth we can conjecture the estimated gap from the CNLSY and Prospects data at the end of spring in first grade.

21. Studies on vouchers, for example, have shown a beneficial effect on disadvantaged Black students, but no such effect on Hispanic students (Chubb & Loveless, 2002). School environmental effects may play a stronger role for Black students, but Hispanic students may also require more attention to language-related concerns (Peterson & Howell, 2002).

22. Similar patterns of slightly more advantaged longitudinal samples versus cross-sectional samples have also been found with Prospects and NELS data (Phillips et al., 1996).

REFERENCE

References

Bali, V., & Alvarez, R. M. (2003). Schools and educational outcomes: What causes the "race gap" in students test scores? Social Science Quarterly, 84, 585-507.

Brooks-Gunn, J., Duncan, G. J., & Klebanov, P. (1994). Economic deprivation and early-childhood development. Child Development, 65, 296-318.

Brooks-Gunn, J., Klebanov, P., & Duncan, G. (1996). Ethnic differences in children's intelligence test scores: Role of economic deprivation, home environment, and maternal characteristics. Child Development, 65, 346-360.

Chubb, J. E., & Loveless, T. (2002). Bridging the achievement gap. Washington, DC: Brookings Institution Press.

Colemen, J. et al. (1966). Equality of educational opportunity. Washington DC: Government Printing Office.

Cook, M. D., & Evans, W. N. (2000). Families or schools? Explaining the convergence in White and Black academic performance. Journal of Labor Economics, 18, 729-754.

Cook, P. J., & Ludwig, J. (1998). The burden of "acting White": Do Black adolescents disparage academic achievement? In C. Jencks & M. Phillips (Eds.), The Black-Wliite test score gap (pp. 375-399). Washington, DC: Brookings Institution Press.

Currie, J., & Thomas, D. (1995). Does Head Start make a difference? American Economic Review, 85, 341-364.

Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Education Policy Analysis Archives, 8. Retrieved December 29, 2003, from http://epaa. asu.edu/epaa/v8nl/

Ehrenberg, R. G., Goldhaber, D., & Brewer, D. (1995). Do teachers' race, gender, and ethnicity matter? Evidence from the National Education Longitudinal Study of 1988. Industrial and Labor Relations Review, 48, 547-561.

Entwisle, D. R., & Alexander, K. L. (1988). Factors affecting achievement test scores and marks of Black and White first graders. The Elementary School Journal, 88, 449-471.

Entwisle, D., & Alexander, K. L. (1994). Winter setback: The racial composition of schools and learning to read. American Sociological Review, 59, 446-460.

Ferguson, R. (1991). Racial patterns on how school and teacher quality affect achievement and earnings. Challenge, 2, 1-26.

Ferguson, R. (1998). Can schools narrow the Black-White test score gap? In C. Jencks & M. Phillips (Eds.), The Black-White test score gap (pp. 318-374). Washington, DC: Brookings Institution Press.

Frankenberg, E., Lee, C, & Orfield, G. (2003). A multiracial society with segregated schools: Are we losing the dream? Report: The civil rights project, Harvard University. Cambridge, MA: Harvard University.

Fry, R. (2003). Hispanic youth dropping out of U.S Schools: measuring the challenge. Retrieved December 29, 2003, from the Pew Hispanic Web site: http://www.pewhispanic.org/site/docs/pdf/high%20school%20dropout%20report-final.pdf.

Fryer, R. G., Jr, & Levitt, S. D. (2003). Understanding the Black-White test score gap in the first two years of school. Retrieved December 29, 2003, from National Bureau of Economic Research Web site, http://www.nber.org/papers/w8975

Gordon, R. A. (1984). Digits backward and the Mercer-Kamin Law: Empirical response to Mercer's treatment of internal validity of IQ tests. In C. R. Reynolds & R. T. Brown (Eds.), Perspectives on bias in mental testing (pp. 357-506). New York: Plenum.

Hanushek, E. (1986). The economics of schooling: Production and efficiency in public schools. Journal of Economic Literature, 24,1141-1177.

Hanushek, E. (1999). Some findings from an independent investigation of the Tennessee STAR experiment. Educational Evaluation and Policy Analysis, 27, 143-164.

Hauser, R. M., & Huang, M. H. (1996). Trends in Black-White test score differentials. Retrieved December 29, 2003, from Institute Research in Poverty Web site, http://www.ssc.wisc.edu/irp/pubs/dplll096.pdf

Hedges, L., & Greenwald, R. (1994). Does money matter? A meta-analysis of studies of the effects of differential inputs on students outcomes. Educational Researcher, 23, 5-14.

Herrnstein, R. J., & Murray, C. (1994). The bell curve: Intelligence and class structure in American life. New York: Free Press Paperbacks.

Jencks, C., & Phillips, M. (1998). The Black-White test score gap. Washington, DC: Brookings Institution Press.

Jeynes, W. H. (2003). A meta-analysis: The effects of parental involvement on minority children's academic achievement. Education and Urban Society, 35, 202-218.

Levin, H. (1996). Economics of school reform for at-risk students. In E. Hanushek & D. W. Jorgensen (Eds.), Improving America's schools: The role of incentives (pp. 225-240). Washington, DC: National Academy Press.

Loehlin, J. C., Lindzey, G., & Spuhler, J. N. (1975). Race differences in intelligence. San Francisco: W. H. Freeman.

McLanahan, S., & Sandefur, G. (1994). Growing up with a single parent. Boston: Harvard University Press, Cambridge, MA.

Meier, K. J., & Stewart, J. (1991). The politics of Hispanic education. Albany: State University of New York Press.

Meier, K. J., Wrinkle, R., & Polinard, J. L. (1999). Representative bureaucracy and distributional equity: Addressing the hard question. Journal of Politics, 61, 1025-1039.

Murnane, R. (1975). The impact of school resources on the learning on inner city children. Cambridge, MA: Ballinger Press.

National Center for Education Statistics. (1995). The condition of education 1995: The educational progress of Hispanic students. [Report 95767]. Washington DC: Author.

National Center for Education Statistics. (1998). The condition of education 1998: The educational progress of Hispanic students. [Report 98013]. Washington DC: Author.

Nation's Report Card. (1999). Results over time: NAEP long term summary tables. Retrieved December 29, 2003, from http://nces.ed.gov/nationsreportcard/tables/Lttl999/

Peterson, P. E., & Howell, W. G. (2002). The voucher programs and the effect of ethnicity on test scores. In J. E. Chubb, & T. Loveless (Eds.), Bridging the achievement gap (pp. 47-73). Washington, DC: Brookings Institution Press.

Phillips, M. (1998). Early inequalities: The development of ethnic differences in academic achievement during childhood. Unpublished doctoral dissertation, Northwestern University, Evanston, Illinois.

Phillips, M. (2000). Understanding ethnic differences in academic achievement: Empirical lessons from national data. In D. Grissmer, & M. Ross (Eds.), Analytic issues in the assessment of student achievement (pp. 103-132). Washington DC: Department of Education, National Center for Education Statistics.

Phillips, M., Brooks-Gunn, J., Duncan, G., Klebanov, P., & Crane, J. (1998a). Family background, parenting practices, and the Black-White test score gap. In C. Jencks & M. Phillips (Eds.), The Black-White test score gap (pp. 103-145). Washington, DC: Brookings Institution Press.

Phillips, M., Crouse, J., & Ralph, J. (1998b). Does the Black-White test score gap widen after children enter school? In C. Jencks & M. Phillips (Eds.), The Black-White test score gap (pp. 229-271). Washington, DC: Brookings Institution Press.

Raundenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications.

Rossell, C., & Baker, K. (1996). The educational effectiveness of bilingual education. Research in the Teaching of English, 30, 7-74.

Wojtkiewicz, R. A., & M. Donato, K. M. (1995). Hispanic educational attainment: The effects of family background and nativity. Social Forces, 74, 559-574.

AUTHOR_AFFILIATION

Valentina A. Ball is Assistant Professor of Political Science at Michigan State University.

R. Michael Alvarez is Professor of Political Science at the California Institute of Technology.

IMAGE CHART 10

Appendix: Sample Attrition

IMAGE CHART 11

Appendix: Sample Attrition

In addition, make sure to read these articles: