One thing I love about #teaching is that you always #learn new things, even when giving a lecture about stuff that you know (or you think you do) very well.
Today a student asked why we should use Student's #t-test at all when comparing samples with equal variances if you can use Welch's variant, which works with unequal variances... that is, why bother checking equality of variances?
Indeed, in the case of equal variances the calculated t statistics would be the same!
This led to a great discussion involving 3 instructors, a bunch of students, and a few simulations in #Rstats which I really enjoyed!
The answer is that Welch's variation also changes how degrees of freedom are calculated, so no they are not equal!
Also, the current recommendation is NOT to check for variance equality (e.g. using Levene's test) as this paper clearly points out, lest you want to increase your Type I error!
https://bpspsychub.onlinelibrary.wiley.com/doi/pdf/10.1348/000711004849222
Gotta go change some slides...
@nxskok Yes, that was the answer I immediately gave... but I did not know the theory about it, and I had never thought about how pre-testing for variance equality could actually be a bad thing
@nicolaromano @nxskok there has been much, much earlier work on this -- https://www.tandfonline.com/doi/abs/10.1080/03610919108812964. If you want to have anything close to reasonable inference, you need to reject the null of equal variances at ridiculous levels like 50%.
@nicolaromano Perhaps the more important question is whether anyone should ever use a t test. A randomisation test is surely always preferable (if you are still happy with p values)🙂
@david_colquhoun Yes, agreed, and indeed in the same course we speak a lot about randomisation tests, bootstrapping etc. However, given how pervasively the t test is used in biology, it's important that students know of it's existence, at they'll come across it when reading the literature
@nicolaromano in practice, unless the variances are definitely not equal, the p-values you get from pooled and Welch tend to be almost identical. The big problem is using pooled when you should use Welch.