I finished reading @lakens ‘s book “Improving Your Statistical Inferences” (freely available here: https://lakens.github.io/statistical_inferences/ #OpenAccess). I originally read it because I was interested in the chapters on effect size and power analysis, but I learned a lot about #stats in general.
As an example, here’s something I never realized about p-values:
“When there is no true effect, and test assumptions are met, p-values for a t-test are uniformly distributed. This means that every p-value is equally likely to be observed when the null hypothesis is true. In other words, when there is no true effect, a p-value of 0.08 is just as likely as a p-value of 0.98. I remember thinking this was very counterintuitive when I first learned about uniform p-value distributions (well after completing my PhD). But it makes sense that p-values are uniformly distributed when we think about the goal to guarantee that when H0 is true, alpha % of the p-values should fall below the alpha level. If we set alpha to 0.01, 1% of the observed p-values should fall below 0.01, and if we set alpha to 0.12, 12% of the observed p-values should fall below 0.12. This can only happen if p-values are uniformly distributed when the null hypothesis is true” (https://lakens.github.io/statistical_inferences/01-pvalue.html#sec-whichpexpect)