On Stillman SHOREs

So, did reducing NZ's alcohol purchase age really hike youth accident rates? SHORE says so:

Now, in the first long-term study, Massey University researchers Dr Taisia Huckle and Karl Parker have found this increased risk has become the new normal.

In the years leading up to the change, drivers aged 18 or 19 had roughly the same chances as those aged 20 to 24 of having an "alcohol-involved" vehicle crash that caused injury or death.

That increased in the years following the change, putting the younger drivers at 15 per cent greater risk in the first six years, then at 21 per cent greater risk up to 2010.

Loyal readers will recall Steve Stillman's work with Stefan Boes showing the opposite: they concluded that there was no increase in crash risk for 18 and 19 year olds after the purchase age change.

So why the difference?

First, the two studies are looking at different things. Huckle et al at SHORE use the ratio of alcohol-involved to non-alcohol crashes as dependent variable. They argue that this helps to control for the rate of non-alcohol-related crashes. They look at how this ratio for 18-19 year olds changes as compared to the similar ratio for older cohorts after the crash. If the ratio increases, that's potentially a form of difference-in-difference analysis that says something about the effects of the law change. They found that the ratio increased for youths relative to older cohorts.

Stillman and Boes instead looked at the number of alcohol-related crashes among 18-19 year olds as compared to alcohol-related crashes among older cohorts. They found no increase in the number of alcohol-related crashes for youths relative to adults once you put in appropriate time trends.

One problem with using ratios as dependent variable, and especially when you're using it in difference-in-difference comparison to another ratio, is that you can't easily tell whether any significant effect of the intervention was due to changes in alcohol-related youth accidents, non-alcohol-related youth accidents, alcohol-related adult accidents, or non-alcohol-related adult accidents. If non-alcohol-related accidents changed for other reasons around the time of the policy change, and if youths responded to that differently than adults, then that could have driven results.

Because the ratio measure isn't clean, it's then harder to talk about the "risk" of an alcohol-involved crash for youths subsequent to the change. You could say that the ratio has changed, but it's hard to say why. Stillman's paper pretty clearly shows that the "why" wasn't an increase in alcohol-involved accidents among 18-19 year olds.

Further, and as Thomas Lumley points out, while the paper says there was no effect on fatalities, each tragic anecdote in the Herald piece involved a fatality; the Herald story nowhere mentioned that SHORE found nothing on fatalities. The SHORE paper said "Lowering the purchase age had no impact on fatal alcohol-involved crashes among drivers aged 18 to 19 years compared with drivers aged 20 to 24 years." I'm a bit curious why the Herald then chose to lead with "Lowering the alcohol purchase age has been linked to a long-term increase in the chance of drunk drivers aged 18 or 19 being involved in car crashes that cause death or injury."

Keeping it 18

New Zealand's reduction in the alcohol purchase age from 20 to 18 did not substantially worsen outcomes for youths. Alcohol consumption did not increase among 18 and 19 year olds subsequent to the change. Consumption, binge drinking, and accident rates among 15-17 year olds seems to have fallen since the reduction in the purchase age as well.

Stefan Boes and Steven Stillman's paper using regression discontinuity design and difference-in-difference measures on the alcohol purchase age, previewed here, is now up as an IZA working paper. Their abstract:

This paper examines the impact of a reduction in the legal drinking age in New Zealand from 20 to 18 on alcohol use, and alcohol-related hospitalisations and vehicular accidents among teenagers. We use both a difference-in-differences approach and a regression discontinuity design (RDD) to examine the impact of the law change. Our main findings are that lowering the legal drinking age did not appear to have led to, on average, an increase in alcohol consumption or binge drinking among 15-17 or 18-19 year-olds. However, there is evidence that the law change led to a significant increase in alcohol-related hospital admission rates for 18-19 year-olds, as well as for 15-17 year-olds. While these increases are large in relative magnitude, they are small in the absolute number of affected teenagers. Finally, we find no evidence for an increase in alcohol-related vehicular accidents at the time of the law change for any teenagers. In an important methodological contribution, we show that one approach commonly used to estimate the impact of changing the legal drinking age on outcomes, an RDD that compares individuals just younger than the drinking age to those just older, has the potential to give misleading results. Overall, our results support the argument that the legal drinking age can be lowered without leading to large increases in detrimental outcomes for youth.

The difference-in-difference measures in particular show lowered consumption among 15-17 year olds.

Finer-grained data on traffic accidents showed no increase with the purchase age reduction. There were more youth hospital admissions for alcohol use disorders subsequent to the law change. But while the increase is large in percentage terms, the baseline risk is low. Boes and Stillman estimate an additional 2.2 to 3.4 alcohol use disorder hospital admissions per month among 15-17 year olds, 2.1 to 2.6 per month among 18-19 year olds, and 1.0 to 1.5 per month among 20-21 year olds in the period immediately following the law change. They note that seasonal variation in admission for flu and asthma are three and twenty times larger than the variation attributed to the reduction in the alcohol purchase age.

They also make an important point on method, and I thank Steven for having explained this one to me slowly; hopefully I've understood it properly. Now recall that they found no effect of the law change on traffic accidents: changing the alcohol purchase age from 20 to 18 did not increase the accident rate among youths. Nevertheless, results from a regression discontinuity design comparing accident rates among kids just under the alcohol purchase age with those just over the alcohol purchase age shows an increase in accidents on reaching the alcohol purchase age after the law change. Why could there be an effect in RDD but not overall? Either the RDD is picking up the effect of the alcohol purchase age on the cohort of inexperienced drinkers who change their behaviour on reaching 18 and experience worsened outcomes, or it's picking up a particular changed behaviour around the time of the birthday. We would have overestimated the effects on traffic accidents for 18-19 year olds by extrapolating from the discontinuity around the 18th birthday.

But, we only know this because New Zealand actually had the policy change so we can compare actual outcomes with those estimated by the RDD. Now, suppose you're in a country that hasn't reduced its drinking age and you want to estimate what the effect of decreasing the drinking age might be. If you ran an RDD looking at the effect of reaching the age of majority on traffic accidents, took the break around the birthday as being the effect of being able to drink, then extrapolated that effect back across the cohort of younger drinkers who would be eligible to drink under a different minimum legal drinking age, you could pretty easily be overestimating the likely effect of a real change in the alcohol purchase age. The RDD picks up the effect of reaching the legal drinking age rather than the effect that would obtain by lowering the legal drinking age.

They conclude:

Overall, our results support the argument being made by groups like Amethyst Initiative and Choose Responsibility (see http://www.choose responsibility.org/proposal/) that the legal drinking age can be lowered without leading to large increases in detrimental outcomes for youth. The current age limit of 21 in the US is higher than in Canada, Mexico and most western European countries. The arguments against lowering the drinking age typically include the idea that even, if a new steady-state with a lower drinking age might be beneficial, the transition to that new steady-state might be very costly. The evidence in our paper from a country with drinking habits very similar to the US suggests that this does not have to be the case.

It is pretty tough to maintain a position that youth drinking is any kind of worsening problem in New Zealand.

Earlier this month, Auckland University's Youth Survey came out. Binge drinking rates among high school students have almost halved since 2001. The NZ Herald highlighted a few other trends from the survey hardly consistent with any kind of growing youth crisis. David Farrar linked up the current survey numbers with prior year figures to show decreases in all measures of youth alcohol consumption.