Tonight I finally submitted my first stats assignment — ten days overdue. The ‘easiest’ of three, they tell me. Univariate exploratory data analysis. Choose about ten variables from a cut-down data set drawn from the Health Survey for England. Calculate the mean and median. Make bar charts and histograms and box-and-whisker plots. Assess normality via skew and kurtosis. And here’s the hard part, for me anyway: develop hypotheses that can be tested using these methods. Wait, what?
For most of those ten days I was grinding my gears over the sheer pointlessness of the task. I don’t care if the distribution of data is ‘normal’ (i.e. Gaussian). If it isn’t, I’ll use a non-parametric test in my next assignment, or I’ll apply an arithmetic transformation, or I’ll consult the literature to identify sensible cut-points to turn interval data into categorical data, or, fuck it, I’ll quote central limit theorem and use a parametric test anyway.
I chose variables to do with social class, material deprivation (independent variables) and smoking uptake, heaviness, and cessation (dependent variables). In my lit review, study after study reported that there’s no difference in uptake by class or education. In many there was no difference in number of quit attempts either, but poor and working class people are much less likely to succeed in quitting.
From a stats point of view, what I’m supposed to care about (in this assignment) was that ‘age of smoking uptake’ had massive kurtosis, i.e. the curve was peaked like a Saturn 5 rocket. I’m supposed to care about that because it affects what tests I can later apply and whether the results they give me, from the limited sample I had at hand, can be taken as a meaningful reflection on reality, i.e. smoking in the population at large.
As Howard Becker puts it, it’s a logic of synecdoche: can we reliably take this sample to stand in for and represent the population?
What I actually cared about was the fact the peak on the graph was around 13 years of age. Forget the stats for a moment and apply some practical intelligence. That, right there, tells you why higher education doesn’t affect rates of smoking uptake — because most people in my sample started smoking in early high school.
Here’s why I don’t care if my data is normal: no matter what test I eventually use, it’s still just a signal. I’m not taking it as gospel even if p<0.000 .. 001. It’s another bit of information I’ll add to the pile along with all the studies I read and my life experience and practical judgment as a practitioner. Bent Flyvbjerg calls this phronesis, i.e. (to simplify quite a bit) good judgment in practice.
This is also why the hypothesis testing pissed me off so badly. I’m supposed to propose hypotheses that can be tested by univariate analysis. ’That the median number of cigarettes smoked in a week will be equivalent to a pack a day’. That’s a univariate hypothesis. It doesn’t compare anything. It’s not ‘that the median cigarettes per week is higher among working class people’ — sorry Dan, that’s bivariate.
Who cares if the first hypothesis is rejected or not? It was totally fucking arbitrary to begin with. I picked an arbitrary value out of the air based on a cultural stereotype, ‘the pack-a-day-smoker.’ But to some people, devotees of null hypothesis significance testing, i.e. the dominant paradigm in quantitative social science, it really, really matters that I pick a hypotheses before I do any tests.
On this view, I’m to perform a pantomime of a scientific experiment, defining my hypothesis ahead of time and then using an appropriate statistical test to falsify it. Of course, falsifying my pack-a-day hypothesis wouldn’t give you any information about what the median was — it just tells you what it wasn’t. So we oh so cleverly phrase the hypothesis in the negative, then we reject it, and Karl Popper doesn’t spin in his grave. YAY!
In practice, nobody but nobody does this kind of testing on univariate data, but as students we’re being drilled in it as a matter of disciplinary socialisation. Because public health has to be Scientific, yo.
Honestly, fuck that. Public health should work. That’s what matters. Who cares if your brother-in-law who’s a research chemist thinks your degree is hard science or not?
If this didn’t scare you off, you might like this great post by Peter Freed: When Central Tendency Junkies Attack, which was inspired by the vitriolic response to this, now rather prescient-seeming post, Jonah Lehrer Is Not A Neuroscientist. TL;DR: Freed argues that statistics should be understood as metaphysics, not a science. This is the philosophy equivalent of a 19th Century gauntlet, thrown down with a cold sneer.