No, the SAT doesn’t just “measure income”

I started this post to refute some specific arguments, but I changed my mind midstream and decided to add a lot more material than I initially envisioned. This is best viewed as being akin to a FAQ (Frequently Refuted Objections – FRO?) relating to the standardized tests and their use in higher education.

SAT and income are not perfectly correlated

The SAT is certainly modestly correlated with parental income, but it is simply not true that the SAT is nothing more than a measure of family income.

I will briefly plot the 2011 SAT reading scores by income level to illustrate that the r**2 is considerably less than one.

There is significant overlap across the entire income distribution:

satv_boxwhiskers.png
Box and whisker plot of simulated test scores
satv_distribution_by_count.png
Simulated distribution by actual test taker counts

Read More »

On sex differences within California public schools

A year or two ago I read an article which demonstrated that sex differences in math and reading achievement are inversely related within and across countries on the PISA tests.  The smaller the male-female math gap, the larger the verbal gap and vice versa.  This tends to support the view that there are innate underlying differences in average abilities and interests between the sexes that strongly influence these patterns.

 

Google Chrome

 

 

Google Chrome

Read More »

On educational attainment rates and income as a causative factor

As I mentioned to Robert VerBruggen in his latest piece on educational attainment and income, I do not believe that economic concerns are a major cause of differences in education attainment rates by income.

I previously analyzed this and related issues using the ELS:2002 data, but I decided to extend my analysis with NLSY97 and clarify my views, now that I have marshaled a fair amount of data to support my arguments.

First

Although the IQ (ASVAB) is an excellent predictor and generally mediates these differences fairly well, there are other systematic differences that are not fully accounted for when you control for IQ.   High SES people, whether measured by income or educational attainment, typically have higher GPAs even with the same test scores.

gpa_by_asvab_income_b5

hsgpa_by_asvab_fed

(these differences would likely to be larger still if I did this as a composite SES index using education, income, occupational prestige, etc)

Read More »

IQ, test scores, GPA, income, and related correlations from NLSY97

Diving into the NLSY97 dataset a bit right now and I thought I’d share some plots pertaining to IQ/achievement tests, income, GPA, educational attainment, and more.   Nothing here is particularly novel, if you’ve looked for this sort of data before, but sometimes it’s nice to have additional independent analysis or alternative presentations of the data.


High school GPA-test score relationships

HS GPA by ASVAB in percentiles (IQ test)

gpa_by_asvab_q25

HS GPA by PIAT (Peabody Individual Achievement Test, IQ test admin. in early childhood)

gpa_by_piat

Read More »

Understanding socioeconomic mobility

Although it may not seem like it at first blush, given the apparently modest correlations, the socioeconomic figures that I blogged about earlier largely agree with the published data on economic mobility.

They are measuring income (or earnings) whereas I am measuring with the ELS SES index (which is an equally weighted average of the respondents own earnings as of 2011, educational attainment, and occupational prestige), but the systematic income differences between classes (however measured) are almost certainly virtually fully mediated by this more comprehensive SES measure.

ELS data, students 2011 SES by 2002 SES of parents

ses_mobility_all_people

The correlation between parent SES and child SES is 0.35.  This may not sound like much, but if you bin child SES and parent SES into quintiles the mobility estimates are very similar to the widely publicized economic mobility estimates.

Microsoft Excel (1)

Microsoft Excel

The average person born at the top of the SES distribution has little chance of ending up at the bottom and vice versa, but there is clearly a great deal of mobility that happens amongst less extreme groupings.

[Note: I didn’t correct these figures for oversampling, so I won’t claim they’re a perfect representation of reality, but they are generally pretty close in practice and they still are good for illustrative purposes.]

Mobility delta (child SES – parent SES) by parent SES

ses_mobilty_all_people

On average, as compared to their parents, high SES people are downward mobile and low SES people are upward mobile.  This may seem counter-intuitive to some, but this makes sense because r < 1, i.e., there is non-trivial mobility, and this must be true for there to be meaningful relative mobility.  Of course, just because there is mobility doesn’t imply that all people have an equal chance of ending up in any place in the SES distribution.

As I mentioned in my last post, there are other differences between groups besides just propensity to end up in particular SES bins and many of these differences are highly predictive of mobility — see test scores, HS GPA, etc.  Indeed, they almost fully mediate outcomes, so talking in terms of “mobility” here can be a bit misleading because relative starting position (parent SES) tells you relatively little about what is likely once you have better information (e.g., test scores, HS GPA, etc).

Read More »

Predicting economic mobility from 10th grade test scores

In my last post I briefly touched upon economic mobility vis-a-vis the link between test scores and subsequent adult incomes.  Because these individuals were still pretty young, just a few years out of college (if they graduated), the earnings correlations were weaker than one might have expected.  Since then I discovered an interesting continuous SES variable (F3SES) in the ELS:2002 data set that is probably a better measure of future earnings or mobility.

F3SES is the average of 3 inputs (2011 earnings from employment, the prestige score associated with the respondents current/most recent job, and educational attainment), each of which is standardized to a mean of 0 and a standard deviation of 1 prior to averaging.

Data users should note that, as of the third follow-up, socioeconomic status may be less-than fully stable for some third follow-up respondents, e.g., respondents with graduate-level education who are just beginning or have yet to begin their careers.  Users should also note that F3SES does not account for the income, occupation, or education of the respondents spouse/partner, and therefore may not be fully indicative of household socioeconomic status as of the third follow-up.

NOTE: While the two versions of the BY family SES composites (BYSES1 and BYSES2) were created by differential assignment of prestige scores based on the 16-category BY occupation variables, F3SES is created by assigning prestige scores based on the 2-digit ONET code associated with the respondents current/most recent job as of the third follow-up.

While I am sure I could derive my own formula to produce a similar composite score, I’ll just use theirs for the time being.


ses_by_test_white_males

ses_by_test_white_and_black_comparison

There is no statistically significance difference between blacks and whites here.

ses_by_test_wba

Asian SES is higher than white SES for most of the distribution, but that’s not statistically significant either.

Read More »

Exploring ELS 2002 data

The NYTimes turned me onto a new data source in a recent article on college graduation rates by SES.  They implied that college graduation rates are better predicted by “wealth” than by the students test scores (10th grade ELS scores taken in 2002).

Google Chrome

Being both curious about the underlying data and somewhat skeptical of the particular claims (or, at least, its interpretation) I decided to investigate it for myself.  Having done so now, I can tell you that it’s a pretty rich data set.  Unfortunately, a few key data points (e.g., SAT scores, HS GPA, etc) are censored or rounded/binned to protect anonymity, but there are still a lot of interesting data there to analyze.


 

Update (6/6/15):

As in my follow-up post on economic mobility, I realized that they actually provided 9th-12th grade high school GPA as a non-continuous variable in the publicly accessible file.  I have updated my post to reflect this new information in a few places!


First point

The parent’s educational attainment is a much better predictor of both test scores and subsequent child educational attainment than economic measures…..

 (Bachelor degree) Attainment rate by test scores, grouped by parent income levels

scatter_with_errorbars_by_income_levels

Read More »

Some additional plots of within school achievement gaps

In a prior post I compared the performance of various ethnic groups directly against each other within the same schools.  That method has the advantage of reducing the effect of racial/ethnic composition from influencing the apparent relationship between each groups performance within school environments.  However, there aren’t always enough of both groups to clear the minimum reporting requirements for student privacy and the school average is a better indicator of “school quality” in some peoples’ minds.  Below I simply took the reported average for all students on the x-axis so that I could compare multiple groups at once.

I do not believe that this is a particularly good way to view the data, since it’s confounded by racial/ethnic composition and sorting by education levels (despite the obvious correlation), but if you do happen to think that the average score is a particularly good measure of “school quality” this approach might be eye opening.

I also plotted this same data for poor and non-poor blacks and whites and (all race/ethnic) scores by parental education level below.  You’ll need to scroll down a ways to see it though.

English

English Grade 2

school_ela_g2

English Grade 4

school_ela_g4

English Grade 6

school_ela_g6

Read More »

Small update to my prior post on concentrated poverty

This is just quick update to my prior post on concentrated poverty.  I re-ran the California test score data at a school-level to compare within school black-white differences in test scores and converted all of the scores data to standard deviation units relative to (above) the non-hispanic mean by school, weighted by the number of test takers.  The pattern can be observed as early as 2nd grade and it is quite consistent for all major/mandatory tests.

 

Black vs White within school comparison

Grade 2 English

bw_ela_g2

 

Grade 2 Math

bw_math_g2

 

Read More »

On concentrated poverty and its effects on academic outcomes

According to a large and growing number of progressives, the achievement gap between “minorities” (especially blacks) and whites can be traced directly to the effects of “concentrated poverty”.  This implies that we cannot compare the outcomes of individual “middle class” blacks to whites of similar income because they don’t have the same amount of wealth, which would allow them to escape their poor neighbors, bad schools, or something along those lines.

Presumably the relationship between the actual neighborhood-level SES, as measured by poverty rates, income levels, education levels, etc, and academic outcomes should look something like this:

prog_model_1

prog_model_2

In other words, this achievement gap is presumably only found in areas of concentrated poverty, but those few families that manage to “escape” these particular bad environments converge on white outcomes or even close the gap entirely.

Having actually studied this data, I can tell you that reality looks more like this:


reality_model_1

reality_2

Put simply, there is no evidence to support convergence.   Broader outcome measures generally show a solidly linear relationship with these measures.  There is also much more overlap in material condition than the picture that most progressives present (curiously they sing a very different tune when they want to talk about these differences in other contexts).    Below I will present some evidence to this effect.

Read More »