QOTO + PEER REVIEW = MODERATOR/GATEKEEPERS?
.
Peer review is at the heart of the processes of scientific journals and all of science.
.
Do we not all understand that peer review is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes won?
.
The peer review process has gatekeepers, not unlike our QOTO moderators -- an online fact-of-life full of problems but the least worst option we have.
.
The most important question with QOTO moderator review is not whether to abandon it, but how to improve it when stumbling blocks arise.
.
A. A specific QOTO issue or
problem has been clearly
identified: Moderators can't
currently read or prevent
advertisements in foreign
languages.
.
B. A QOTO community vote
rejected a rule proposed to
address the problem.
.
C. What next? What can
QOTO do -- what are we
willing and able to do -- to
mitigate harm caused by
doing nothing in response
to a clearly identified
problem?
.
Moderator/gatekeeper review is a flawed process; but it is likely to remain central to QOTO because there is no obvious alternative,
.
Going forward, QOTO will grow by learning lessons the hard way, right? And QOTO needs to figure out to do better, right?
.
NOTE: This post paraphrases Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006 Apr;99(4):178-82. doi: 10.1258/jrsm.99.4.178. PMID: 16574968; PMCID: PMC1420798; see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/
.
QOTO = Question Others to Teach Ourselves?
@chikara @freemo I am coming late to this discussion, and I mention what follows with fear and trepidation. I'm curious whether this kind of system has been considered.
I have occasionally admired the results of Slashdot's moderation and metamoderation system even though I certainly heard participants complain about it. I never created an account there and always read anonymously, but I did find the quality of the comments that percolated to the top was generally fairly good even when the site was far more active than it is now.
I don't mean this to be flippant. You guys are talking in terms of peer review. For all its numerous faults, peer review has high ideals. Let experts judge noteworthy work. The Slashdot moderation system is hardly all that.
I am also unsure how exactly a moderation and metamoderation system would translate into the context of a Mastodon instance.
Nevertheless, from the perspective of an outside observer Slashdot did seem to select a fairly reliable set of moderators from a large crowd of mostly anonymous individuals some of whom were openly pushing specific agendas. And with the metamoderation system they took it a step further to answer the question: quis custodiet ipsos custodes?
As I understood it the system tended to select moderators that were long-time readers who did not have a history of negatively-rated comments. That was basically the standard of "expertise" required of a moderator, although there were some instructions to encourage moderating based on how the comment contributed to the discussion, not whether the moderator agreed with it or not.
In any case, I suspect QOTO has attracted a specific kind of participant because of the way it describes itself and the tools it makes available. It would certainly be a shame to see that drowned out in the noise as it grows.
All that said: growing pain is the best kind of pain there is! Congratulations.
