Nicola Romanò @nicolaromano@qoto.org

How do we make sense of the body of scientific literature that is growing explosively to the point where no individual could read all the relevant papers, and is contaminated with fraudulent and LLM-generated papers? I think that science is not currently equipped to deal with this, and we need to.

I think a critical part will be post-publication peer review. With such rapid growth and time pressure on scientists, pre-publication PR cannot maintain sufficient standards so we need a way to make the informal networked review (journal clubs, conference chats etc.) more transparent and shared.

We also need ways to summarise and make connections between many related papers. I know that many people are hoping that LLMs will step up into this role, but right now that seems very risky and I don't see that changing any time soon.

LLMs are too distracted by surface level presentation, and can be manipulated at scale by iterating over multiple presentations until the LLM summarises it in the way you want it to. In addition, they're known to have problematic biases, and it's unclear if this can be fixed.

I think we need to be experimenting with ways to distribute the work of summarising and making connections between papers, and aggregating that into a collective understanding. An individual can't read all the papers, but collectively we can and already are. We just need ways to integrate that.

In principle we could do this with a post-publication peer review system that allows reviewers to annotate with connections, e.g. in reviewing paper X you create a formal link saying that part of this paper is similar to paper Y, or uses the same technique, etc.

One issue is that these annotations might become corrupted or manipulated in the same way that papers, journals and peer review have been. How do we fix that? It's not ideal, but one option might be some form of trust network: I trust X, they trust Y and thereore I (to a lesser extent) trust Y.

This would mean our summary or evaluation of the literature would depend on our individual trust network. But, this isn't a bad thing in principle. Diversity of opinion is good: there shouldn't be one definitive reference summary because that's a single point of failure and target for manipulation.

All these ideas require experimentation, and both technology development and a cultural shift towards collectively taking responsibility for doing this work. I think we need to do it and would love to hear others' ideas about how to do it and how to convince everyone that we need to.

#science #metascience #academicchatter

science needs to own its digital structure : that’s new, non-extractive, open publishing, data repositories, and new modes of peer review. But the glue that’s needed to tie that all together is communication tools owned and designed by scientists themselves. I really believe Bonfire can give us that …

if you can support their crowd funder, please do!

#OpenScience #science #OpenScienceNetwork

https://www.indiegogo.com/en/projects/bonfire/community

Universities should boycott the sleazy organizations that make money by ranking universities according to questionable criteria using non-transparent methods.

And they're starting to do it! The Sorbonne has just announced that it won't give data to the Times Higher Education rankings anymore. Columbia University and Utrecht University have also quit, as have the medical and law schools of Harvard and Yale.

700 research organizations, funders and professional societies have signed the Agreement on Reforming Research Assessment in favor of making scientific research, data, methods, and educational resources transparent, accessible and reusable by everyone without barriers.

But we're a long way from getting there!

https://scroll.in/article/1087997/why-the-worlds-top-universities-are-abandoning-the-ranking-system

arXiv will no longer accept review articles and position papers unless they have been accepted at a journal or a conference and complete successful peer review.

This is due to being overwhelmed by a hundreds of AI generated papers a month.

Yet another open submission process killed by LLMs.

https://blog.arxiv.org/2025/10/31/attention-authors-updated-practice-for-review-articles-and-position-papers-in-arxiv-cs-category/

My work #EMR at now has integrated #AI that summarizes a patient's chart whether I want it to or not. This week it told me the wrong reason for admission, the wrong hospital course, and the wrong medications as compared against the human-written discharge summary. To review it and find the error took 3 minutes; to document the error and report it took another 10.

Anchoring bias exists. What we read stays with us, truth or lie, influencing decisions.

And I can't turn it off.

#LawsuitBait

I'm now also looking for a postdoc with strong Bayesian background and interest in developing Bayesian cross-validation theory, methods and software. Apply by email with no specific deadline (see contact information at https://users.aalto.fi/~ave/).

Others, please share

#bayesian

#rstats folks should think about supporting our friends at the #Python Software Foundation, who turned down a $1.5M NSF grant rather than cave to the administration's inquisition against diversity, equity, and inclusion. ❤️

https://pyfound.blogspot.com/2025/10/NSF-funding-statement.html