🇳🇱
I have been thinking about writing a webapp that would enable QOTO to become the first "open, distributed, scientific journal".. I am still thinking about how I am going to go about it but the idea would be a scientific journal that anyone, regardless of credentials or pre-approval, can submit articles and peer-review and ultimately the acceptance or rejection of an article will happen without a centralized authority.
In all liklihood acceptance will be "fuzzy" in that it will be up to the reader to decide if the groups which approved of the article through peer review have sufficient credentials that you trust them. So the key will be in designing a way to make that work.
Ideas are welcome of course.
@freemo Scientific publishing is already decentralized. Publishing decisions are made by journals and for most studies there are many journals to chose from.
Especially if you count journals without reputation, which QOTO would (initially) be, there is more than enough choice. Anyone can start a journal. https://pkp.sfu.ca/ojs/
Coding a new journal system is a fair bit of work.
@GrassrootsReview Many centralized journals does not equate to decentralization. The whole point of a decentralized framework is to remove the authority and thus any need for consideration of a journals integrity (as there are no journals).
@freemo For me decentralized journals do mean that the system is decentralized.
Reputation is important. Otherwise it is just blogging, which is decentralized, but not particularly suitable for scientific progress.
But do write up how your system would work and which problem it would solve.
@GrassrootsReview Reputation is important, but not of the journal, of the people publishing and those doing the peer review.
Decentralized journals dont exist, only a collection of centralized journals, not the same thing. Each journal is centralized and largely independent from each other.
In a proper decentralized systems the article writers and peer reviewers would have a reputation that isnt tied with or verified by any journal acting as an authority but rather in a decentralized way.
Yes I agree a proper writeup on the mechanics would be key to really have a meaningful discussion of my plans.
@freemo In a good system the reputation of the authors is not important. My colleagues will be willing to read my blog posts, but new people and outsiders need the support of the reputation of the journal.
The journal and the editors are named and can build up a reputation.
Reviewers are normally not named. Requesting them to do so makes it a lot harder to find reviewers because that means writing reviews affects your reputation and which would require putting in 10 times as much work.
> In a good system the reputation of the authors is not important. My colleagues will be willing to read my blog posts, but new people and outsiders need the support of the reputation of the journal.
That doesnt add up to me. Typically the journal decides who can or can't get the final say on peer-review. So yes, the people submitting and authoring the peer review absolutely matters. The journal just acts asa a proxy to decide who can or can not make the call on peer-review.
> Reviewers are normally not named. Requesting them to do so makes it a lot harder to find reviewers because that means writing reviews affects your reputation and which would require putting in 10 times as much work.
Right, my system would allow reviewers to remain anonymous while still allowing them to gain a reputation. You might know two reviewers are the same person, but you wont know who that person is.
The problem here is that while I agree reviewers should have some level of anonymity, at the same time the general public should be able to review the peer-review process itself and judge it on an article by article basis, as well as judge the reputation of the reviewer (without knowing their identity, so based solely on their past quality of work reviewing).
@freemo Peer review helping outsiders is explained here in more detail. It adds up. http://variable-variability.blogspot.com/2014/01/peer-review-helps-fringe-ideas-gain.html
Normally an editor, not the journal decides on which manuscripts are published and who becomes reviewer. The editor-in-chief ("journal"?) typically decides which editor does which manuscript. The peer reviewers are advisors of the editor, but do not decide.
Publishing the peer review reports is an idea I like and only the arguments should count. The editor as the decider is known.
@GrassrootsReview Right, the point here is there is a centralized group (in this case the editors and peer reviewers collectively) who decide reputations and what passes and what doesnt, and generally obscures the peer-review process publicly. Thus not just centralizing the process to a select few at the journal, but ultimately obscuring it from public scrutiny as well.
The point of decentralization is openness, to allow the consumer of the article to have the power to judge the reputation by opening up the processes and giving them the information needed to do so.
@freemo Just start you own journal if you prefer editors and reviewers who have less expertise. The consumer can decide to read it.
Climate "skeptics" have been there before and have started something that looks like a scientific journal to fool the public and non-science journalists. http://variable-variability.blogspot.com/2014/02/global-warming-solved-in-open-peer.html
Interestingly I just got a follow by some Social Darwinian COVID deception account. The same MO as the climate "skeptics", just another way to kill people.
@GrassrootsReview I never said anything about prefering a journal where edirors or reviewers have less expertise. I am talking about a system that is open and transparent, hopefully this should result in **more** expertise, not less.
The last thing an open and transparent process would do would be to create climate "skeptics" articles that are considered credible, since it the system would require a scientist to prove their expertise openly. In fact quite the opposite, such an open and transparent, but high reputation/expertise, decentralized system ideally should gain trust not just from traditional scientists but also people like climate change deniers, and this hopefully should build trust with the scientific community once again.
> That problem is solved by starting you own journal in the decentralized system we have.
It is not, starting my own journal would put me in charge, and the journal would not be "decentralized", like all journals it would be centralized (in this case me being the central point of authority for my journal).
The problem with centralized systems, such as journals, is that I am in control and can obscure whatever parts of the process I wish. If I happen to take a bribe (as rarely but occasionally happens) there are no good checks and balances to flush that out. I might get caught, I might not. Generally its not a huge issue with journals, but it has happened.
The distrust in science, which seems to come from across the political spectrum, doesnt come from no where. These centralized and closed processes create suspicion and ruin the communities trust in journals. The climate deniers are wrong in their denial of climate, but they are right regarding the lack of transparency and the power distribution that ultimately fuels their suspicion.
It isnt about quality of science so much as the transparency and the ability for individuals to have the means to judge that quality, which only comes when the system is open, both in terms of open access, and open processes.
@freemo Everyone can start a journal, not just you. So it is perfectly decentralized.
If you start your own journal or in some decentralized system contribute to the quality of the scientific literature less frequently, it is (initially) hard to judge whether you do it well. Bribes would thus be more effective.
Trust in science is high & science is becoming more open. The world is increasingly unequal and authoritarian and authoritarians do not like their power being constrained by reality.
@GrassrootsReview decentralization is not "everyone can start a centralized and walled off community".. that isnt in any sense what decentralization is. For something to be decentralized there has to be some sense of interoperability and organization.
I really dont know why you insist that having more than one centralized entity is somehow equivalent to decentralization.
> or in some decentralized system contribute to the quality of the scientific literature less frequently, it is (initially) hard to judge whether you do it well. Bribes would thus be more effective.
Bribes are more likely, of course, if you dont have a reputation yet.. thats the whole point of a decentralized system specifically building reputations through webs of trust. It isnt until I've proven that i am not susceptible to bribes than my opinion and publications hold a high trust value in the first case.
@GrassrootsReview I just realized your accounts description is almost exactly what im advocating for with an open decentralized peer review process. After reading it and how it is almost word for word what im arguing for here I am rather surprised you are arguing against it.
To quote you:
Let's bring the quality control of scientific articles back to the scientific community with open post-publication peer review independent of scientific journals
@freemo If that was what you were proposing, then I like it. 😎
But it is not just another journal (it does not publish, it only reviews). It is a journal-independent review. Starting a new journal (system) will not break the power of the publishers.
It has editors, who moderate the debate. Named journals and named editors who build up a reputation and make it possible to have anonymous comments without becoming YouTube comments.
It is better for science by being post-publication peer review.
@freemo You wrote: "Right, the point here is there is a centralized group (in this case the editors and peer reviewers collectively) who decide reputations and what passes and what doesnt, and generally obscures the peer-review process publicly."
That problem is solved by starting you own journal in the decentralized system we have.
If quality of science mattered we would not have climate "skeptics", anti-vaxxers or COVID deniers. Quality is not a good predictor of their memes.