Hubert Plisiecki @hplisiecki@qoto.org

@annaleen With all due respect, if it would be possible to infer correctness from the overall "vibe of the response", machine learning engineers would have a hell of an easy job when checking if a response is a misinformation. Unfortunately that is not the case.

@uebernerd There are situations, very specific ones but still, where conducting a high powered study is systemically impossible (i.e. require a lot of resources that are hard to justify to grant givers). Wouldn't conducting a lot of low-powered studies by different teams and reanalysing the results in a meta-analysis be one of the ways to get some insight into the subject of such studies?

@uebernerd What do you think about the Journal Meta Psychology stand on the matter: That as long as the underpowered study is well preregistered, it can be a useful addition to meta-analyses?

@david_colquhoun Indeed, the main aim of the verification is the incorrect rounding of the statistic or pure fabrication. Checking earlier steps would be way more problematic as they are not usually directly reported.

@david_colquhoun
I made sure to make as few changes to the original code as possible, and have recoded all original tests. In terms of the standard deviations and errors, it actually doesn't extract them;

It extracts the value of a test statistic, and the degrees of freedom and recomputes the p-values on that basis.

I have just finished rolling out the Python version of the R package 'statcheck", created by prof. Michèle Nuijten.

Statcheck can be used to extract and analyze statistical results from scientific articles.

Currently working on adding the option to extract tables.

If anyone would like to check it out, here is the repo:
https://github.com/hplisiecki/statcheck_python?fbclid=IwAR30Y1Gxn0k9bK3rKWTmGA6tHi-X_lrBKnvvLz0v9IMOmF8QXV29g0LYCRI

#python #psychology #statistics
#openscience

Another option would be limiting the toots from specific Mastodon instances.

Would it be possible to modify the toot feed based on user recomendations? For example add an option to see "these kinds of toots less". I know that Meta does these kinds of things in a centralized manner - limiting controversial posts (info from Lex Fridman's podcast with Zucc).
Of course to do that on the level of "type of content" would require a personalized recommendation model, but perhaps an option to just limit the ammount of toots from a specific profile would be possible?

Not the type of silver lining I was looking for

@nomi Except that in order to buld, calibrate and so on, you first need the motivation to do it. That motivation has to stem from recognizing the relevance of the sought distinction and wanting to find out.

@nomi My toot related to the possibility of making such a tool. Its existence would have meant mean that it was differentiable already.

@nomi *if not

@nomi What if you could devise a tool to distinguish these scenarios, or at least start a project to develop them?
Would their still be irrelevant in the current moment.
And if yes, then in what instances would you be sure that such a tool cannot be developed?

@Riedl I like that it hangs on to its premises. Prompt: "Can we find material evidence for Angels?"