On the popularly reported black implicit association test (IAT) results

Recently the media and various friends and family have been asserting that implicit association tests (IAT) “prove” that whites are biased against blacks and that this presumably substantially explains the racial disparities in police shootings.

Race_630_Racist2

Since I am skeptical about the racial angle in police shooting, the validity of measures like IATs, and of received wisdom in general, I thought I would take a look at “Project Implicit” to better understand it.  The raw data for these results is available in SPSS format on OSF.io (albeit at >2GB) so I downloaded the data and performed some analysis in R.

Here are a few things I can say:

1: The reported averages by group (e.g., ethnicity, gender, political views, etc) hides a lot of variance within groups and overlap amongst groups.

black_iat_us_only_by_race_ethn_all_iat_count

The typical standard deviation is ~0.4 for every group with a reasonably large N.

black_iat_us_white_only_by_religiousity

black_iat_us_only_white_only_by_politics

[Note: 1 = strongly conservative, 7 = strongly liberal.  N for 7 is fairly small]

iat_white_by_age_level

iat_white_by_sex

2: The differences between the averages are typically very small by comparison to the variance in all of these groups

The only arguably exception to this is the target group (as in, blacks in this case), but even then this presumably implies that a long of blacks are strongly biased against black people.

3: The measured differences in the United States between whites, asians, hispanics, and other major ethnic groups are pretty negligible.

black_iat_us_only_by_race_ethn_all_iat_count

If these tests are measuring something “real” and important, the reporters ought to observe that non-hispanic whites are broadly inline with every other group, save for blacks.

4: The patterns appear to hold up internationally

iat_white_both_rez_and_cit iat_white_by_country_primary_residence iat_white_by_primary_citizenship

iat_black_non_us_citizen_residents_by_race_ethnic_group

iat_black_amongst_non_us_by_raceethnic

These patterns appear internationally and do not appear to be notably stronger amongst non-hispanic whites in the US than are in generally European countries. Other ethnic groups, both inside and outside the US (various configurations), also have this “bias” (again, save mainly for blacks/africans).  Non-Hispanic whites appear to have a slightly stronger “white” bias than other ethnic groups, but I would hardly say that that is strong evidence for even marginally higher anti-black bias.  As in, it’s more likely that groups prefer their own and certainly find them more familiar, other things being roughly equal.  I would bet that acculturation of minority groups in various majority populations likely makes them more familiar with the faces they encounter more regularly, providing that experience is not a generally negative experience.

5: Said bias appears to correlate positively with “diversity”

iat_white_by_pct_county_black_binned

Non-hispanic whites reporting to live in counties (not just the state) with large black populations are more likely, not less likely, to exhibit this bias.

iat_white_by_pct_nhw_finer

Likewise, “whiter” counties are less likely to exhibit this “bias”.  I would think this would run counter to the narrative of the people reporting this stuff, i.e., presumably whites with less interaction with blacks should harbor stronger biases, not weaker biases.

iat_black_scatter_by_pct_nhw_means iat_black_scatter_by_pct_county_black_means

iat_black_scatter_faceted_pct_black_min100

iat_black_faceted_scatter_state_pct_nhw_min100

iat_black_scatter_faceted_state_pct_black_min100

iat_black_scatter_faceted_by_state_black[Note: East Asians, Hispanics, and other groups also demonstrate more “bias” the larger the black population and/or the smaller the non-hispanic white population]

See here for: IAT (W-B) amongst non-hispanic whites PDF (long) density graphs

6: Similar “bias” exists against “Asians”

iat_asian_us_only_race_ethnicity

asian_iat_white_us_by_ed_level

asian_iat_us_white_only_by_age_level

iat_asian_us_white_only_by_politics

asian_iat_white_by_country_res

Asians have higher income, higher education, significantly lower imprisonment rates, higher life expectancy, better average grades, and so on and so forth for most measurable statistics.  A certain breed of progressive even calls them a “model minority”.  They surely also have fewer interactions with the police despite this IAT “bias” that is presumably every bit as strong against them (and I would bet that they are under-represented nationally in police shootings and in crimes against the police, i.e., based on my recollection of summary statistics from my earlier review here).

7: The tests have real measurement issues

iat_white_by_number_of_iats

People that report taking more IATs have “significantly” less bias.  I suspect there is a real training issue here (which calls into question the whole pursuit)

iat_white_by_iat_order

Likewise, there is variance (which they imply is significant in other contexts) depending on the order of the test.  If all groups are observed at a high rate this might not be an issue, but when the N is small this is likely to (randomly) skew the results further, i.e., depending on which order the test takers are randomly assigned.

8: The faces chosen surely influence the results

I have not reviewed the literature, but, other issues aside, they only use a small number of faces to reach sweeping conclusions about race bias/association.  Are these faces and expressions truly representative of the broader black or white population so that we can draw sweeping conclusions about normal interactions in the real world?  I would bet that there is considerable variability depending on the facial structure, gender, expression, facial hair, etc present on the face presented (I don’t think they had this level of detail in their data files, but it’s possible I missed it), not to mention things that they do not even show like clothing, observed behaviors, etc.

black_iat_faces

asian_iat_faces

Some additional perspective on the heat maps by state

project_implicit_heat_map

Google Chrome 4 Google Chrome 2 Google Chrome 3

Google Chrome

[Note: some sample sizes with some of the minority groups are small in some of these states (I should have filtered them first!), so I wouldn’t read too much into it them other than, perhaps, to note broader patterns in this data]

Advertisements

One thought on “On the popularly reported black implicit association test (IAT) results

  1. excellent job breaking down the data – you should get paid for this! or get a pub out of it. (either a publication or a trip to the pub – whatever works:) especially well done looking at overlapping distributions – if more people understood that concept there would be less animosity when discussing group differences in more volatile areas!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s