Created: June 08, 2021
Modified: June 08, 2021
Modified: June 08, 2021
the null hypothesis is always wrong
This page is from my personal notes, and has not been specifically reviewed for public consumption. It might be incomplete, wrong, outdated, or stupid. Caveat lector.- Andrew Gelman believes that in certain areas of research, like the social sciences, everything is connected.
- "I’m not expressing pessimism or extreme Cartesian doubt, I’m just saying that in the sorts of problems I work on, everything is statistically dependent on everything else. For example, we’ve modeled responses to “How many X’s do you know?” questions. Older people know more older people, younger people know more younger people. Women know more women, men know more men. If we have enough data, we’d find that the differences between old and young are different comparing men and women. Of course the differences will differ—that is, of course there are two-way interactions, if we were to look at the entire population. There’s no way that the differences would be identical. Similarly there is geographic structure in who you know. People know other people who live closer to them. But the relevant distance scale is different in NY than in LA. But it’s not just travel time either. And of course the differences by distance are different for different age groups. They have to be: how could they be exactly the same? And so on. I agree that some of these differences are small but I don’t think any are exactly zero, nor do I think it’s such a good idea to work on statistical methods that are based on finding the zeros or finding the nonzero interactions. Cos the trouble is, that in the worlds where I work, there’s a continuum of dependences. It’s not like there are a few huge things and a bunch of tiny things." - Whither the “bet on sparsity principle” in a nonsparse world? « Statistical Modeling, Causal Inference, and Social Science (columbia.edu)
- When is the null hypothesis true?
- Sometimes we are evaluating something other than an 'effect size'. For example, if we're testing whether there is life on Mars, we could think of this as asking 'how much life is there on Mars?', and the answer 'zero' is quite plausible.
- In the physical science, effect sizes really can be zero or infinitesimal. A drug might really have no effect on cancer cells. A butterfly flapping its wings on Earth might really have no effect on Mars.