Predicting economic mobility from 10th grade test scores

In my last post I briefly touched upon economic mobility vis-a-vis the link between test scores and subsequent adult incomes.  Because these individuals were still pretty young, just a few years out of college (if they graduated), the earnings correlations were weaker than one might have expected.  Since then I discovered an interesting continuous SES variable (F3SES) in the ELS:2002 data set that is probably a better measure of future earnings or mobility.

F3SES is the average of 3 inputs (2011 earnings from employment, the prestige score associated with the respondents current/most recent job, and educational attainment), each of which is standardized to a mean of 0 and a standard deviation of 1 prior to averaging.

Data users should note that, as of the third follow-up, socioeconomic status may be less-than fully stable for some third follow-up respondents, e.g., respondents with graduate-level education who are just beginning or have yet to begin their careers.  Users should also note that F3SES does not account for the income, occupation, or education of the respondents spouse/partner, and therefore may not be fully indicative of household socioeconomic status as of the third follow-up.

NOTE: While the two versions of the BY family SES composites (BYSES1 and BYSES2) were created by differential assignment of prestige scores based on the 16-category BY occupation variables, F3SES is created by assigning prestige scores based on the 2-digit ONET code associated with the respondents current/most recent job as of the third follow-up.

While I am sure I could derive my own formula to produce a similar composite score, I’ll just use theirs for the time being.

---

There is no statistically significance difference between blacks and whites here.

Asian SES is higher than white SES for most of the distribution, but that’s not statistically significant either.

Exploring ELS 2002 data

The NYTimes turned me onto a new data source in a recent article on college graduation rates by SES.  They implied that college graduation rates are better predicted by “wealth” than by the students test scores (10th grade ELS scores taken in 2002).

Being both curious about the underlying data and somewhat skeptical of the particular claims (or, at least, its interpretation) I decided to investigate it for myself.  Having done so now, I can tell you that it’s a pretty rich data set.  Unfortunately, a few key data points (e.g., SAT scores, HS GPA, etc) are censored or rounded/binned to protect anonymity, but there are still a lot of interesting data there to analyze.

---

 

Update (6/6/15):

As in my follow-up post on economic mobility, I realized that they actually provided 9th-12th grade high school GPA as a non-continuous variable in the publicly accessible file.  I have updated my post to reflect this new information in a few places!

---

First point

The parent’s educational attainment is a much better predictor of both test scores and subsequent child educational attainment than economic measures…..

 (Bachelor degree) Attainment rate by test scores, grouped by parent income levels

Exploring college completion data

Today I am exploring college completion data from IPEDS which provides detailed information on major choice, admissions related information, and the like.

---

Amongst universities requiring test scores and reporting them, 6-year graduation rates by estimated SAT score.

Correlations

• White: 0.78
• Asian: 0.69
• Black: 0.70
• Hispanic: 0.71
• Women: 0.79
• Men: 0.82
• Total: 0.82

A quick post on crime and test scores

To further extend my California test score series, I decided to compare crime rates at a county level to test scores (English Grade 11) for all test takers.

Violent Crime Rate

[Un-weighted correlation: -0.38]

Homicide Rate

[Un-weighted correlation: -0.40]

Sex differences within schools

I have long been of the impression that most of the gender gap in academic achievement is skewed towards lower SES groups and URMs, that much of it has much to do with a comparative lack of male engagement in primary and secondary school, especially with respect to home work compliance, so I decided to exploit my prior efforts to try to assess the accuracy of this view.  Of course this data isn’t broken out by race or SES per se, but the right side on the x-axis is generally higher SES and more white and asian (whereas the left side is generally more black and latino and generally lower SES).

English

English Grade 2

English Grade 4

Some additional plots of within school achievement gaps

In a prior post I compared the performance of various ethnic groups directly against each other within the same schools.  That method has the advantage of reducing the effect of racial/ethnic composition from influencing the apparent relationship between each groups performance within school environments.  However, there aren’t always enough of both groups to clear the minimum reporting requirements for student privacy and the school average is a better indicator of “school quality” in some peoples’ minds.  Below I simply took the reported average for all students on the x-axis so that I could compare multiple groups at once.

I do not believe that this is a particularly good way to view the data, since it’s confounded by racial/ethnic composition and sorting by education levels (despite the obvious correlation), but if you do happen to think that the average score is a particularly good measure of “school quality” this approach might be eye opening.

I also plotted this same data for poor and non-poor blacks and whites and (all race/ethnic) scores by parental education level below.  You’ll need to scroll down a ways to see it though.

English

English Grade 2

English Grade 4

English Grade 6

A brief analysis of suspension rates in California public schools

Since I have heard more than a few complaints about disparities in school suspension rates, I thought I’d extend my prior analysis of data from California public schools to better understand the patterns here.

UPDATE: I realized after the fact that I botched the asian rate data a bit because California reports Filipinos as a separate group for some strange reason, i.e., they’re not included in the reported suspensions/numerator, and my denominator uses the normal federal/OMB definition, which includes doesn’t put filipinos in a separate category.   Filipinos are about 1/4th of the (federal) Asian category in CA schools and probably have a higher suspension rate than east asians (which is extremely low).  I don’t think it’d alter the between group differences all that much, especially not in ordinal terms, but I don’t feel like re-doing these all plots right now, so keep that in mind.

Out-of-school suspension rate (“(OOOS”) by district (weighted)

OOSS by district (weighted), y-axis as log-10

Observation: There are clear racial/ethnic patterns here that cut across many different school districts.

Standardized tests correlations within and between California public schools

To exploit some of my prior work with California’s test score data, I decided to extend this analysis to SAT, ACT, and AP scores in the state of California, i.e., to compare the relationships of these different tests within and between schools.

Notes/Caveats:

• To achieve more stable results with schools with small numbers of test takers I averaged 2-4 years worth of test score data together.
• Unfortunately, AP results are not broken out by subject (some are much harder than others and there are apt to be different test taking patterns at different sorts of schools)
• SAT/AP/ACT data is not available by race/ethnicity

ACT vs SAT (r=0.97)

[It’s almost like they’re testing the same construct…. :-)]

SAT by English grade 11 (r=0.84)

Some quick plots pertaining to various “Asian” ethnic groups

Back in my earlier post on the effects of concentrated poverty, I mentioned that California provides test score data for various specific “Asian” ethnic groups (e.g., Chinese, Japanese, Korean, Hmong, Indians, etc) and that I might update my analysis with that information later.

To aid interpretability across multiple tests, I first converted all average scores into standard deviations above the non-hispanic white mean (both are weighted for the number of test takers in each school district to better approximate the actual individual distributions) and then plotted these as box plots for detailed ethnic groups, poverty status by race/ethnicity, and education levels.  All of these box plots are also weighted by the number of test takers in each group.

X = 0 should correspond to the weighted white mean across school districts in California (which ought to be very close to the individual white average state wide), ergo groups or parts of groups (e.g., IQR) that are on the right hand side are generally outperforming the white mean and vice versa for groups on the left hand side.

English Grade 2

Math Grade 2

What “blank slate”?

James Thompson of Psychological Comments blogged on a huge meta-analysis of 50 years worth of twin studies published in the journal Nature. (Full text copy here).  Although the study covered a wide range of phenotypes, several of them have particular relevance to intelligence, academics, and occupational outcomes.  The authors of the study also published an interactive website, MaTCH, where more detailed statistics can be browsed.

I screen captured traits some traits relevant to academic and occupational outcomes below:

Structure of Brain

Higher level cognitive function (general intelligence)