I just finished Stuart Ritchie's jaw-dropping book "Science Fictions" which exposes the deleterious effects of fraud, bias, negligence and hype on the constitution of scientific knowledge. Of course this is a a deplorable but all-too-familiar observation, however the book brings a new level of detail. (1/3)
A particularly enlightening insight is the estimation of the prevalence of fraud (e.g. how often do biologists fake the figures in their papers?), publication bias, p-hacking, and even numerical errors in published papers... There are public records (such as Retraction Watch), very clever tools (such as statcheck http://statcheck.io/), and tests (e.g. the GRIM test https://en.wikipedia.org/wiki/GRIM_test) (2/3)
It's reassuring to know that despite the perverse incentives, despite the crooked publication system, science does actually contain the tools to evaluate its own weaknesses, and maybe to heal itself. (3/3)
@leovarnet I'm more pessimistic. Do you know Elisabeth Bik's work on images, especially in the biosciences? I'm worried about AI generating fakes. Real scientists, a minority of those holding the title, should find out themselves and go on.
@Waldemar My opinion is that "real scientists" already know themselves, in a sense: for example, experts of a given field are able to predict which studies will replicate or not. The problem is for the general public and the constitution of a common knowledge...
