#researchassessment

News from the @DORAssessment page:

Brief report about a project aimed to test the effectiveness of multimedia formats for educating researchers about scholarly metrics
https://sfdora.org/2025/04/07/introducing-two-educational-videos-on-the-h-index-the-pedagogical-potential-of-multimedia-for-teaching-about-scholarly-metrics/

The team used multiple strategies to develop content and format of the videos. I found it an interesting brief read for the planning of educational resources.

If you have not heard about problems with the #hIndex: Another reason to watch. #AcademicChatter

#ResearchAssessment #REF2029 #VideoTaxonomy

vs in #ResearchEvaluation? A sharp comparative study shows how Brazil’s high-stakes, performance-based model contrasts with the Netherlands’ strategic, decentralized approach.

https://doi.org/10.1093/reseval/rvaf013

Takeaway: Evaluation isn’t one-size-fits-all - context matters.

#ResearchAssessment #SciencePolicy #HigherEducation #ResponsibleMetrics #AcademicEvaluation

The DORA team invites you to celebrate the 12th birthday of @DORAssessment

This will be accompanied by the launch of a new publication: “A Practical Guide to Implementing Responsible Research Assessment at Research Perfoming Organizations"

https://sfdora.org/2025/04/14/launch-of-a-practical-guide-to-implementing-responsible-research-assessment/

#DORA #ResearchAssessment #OpenScience #OpenAccess #CoARA

Episode #11 of the #GraspOSChats series of interviews is out!

Kumar Guha and Samuel Scalbert from @inria tell us about their work on the #GraspOS pilot on #ComputerScience and share insights into the development of the #OpenScience Assessment Framework (OSAF) and how it supports #ResearchAssessment reform.
Read the full interview: https://graspos.eu/graspos-chats-11-get-to-know-kumar-guha-and-samuel-scalbert
Check out the GraspOS Chats collection of interviews to learn more about the people behind the scenes: https://graspos.eu/graspos-chats

ResearchFish Again

One of the things I definitely don’t miss about working in the UK university system is the dreaded Researchfish. If you’ve never heard of this bit of software, it’s intended to collect data relating to the outputs of research grants funded by the various Research Councils. That’s not an unreasonable thing to want to do, of course, but the interface is – or at least was when I last used it several years ago – extremely clunky and user-unfriendly. That meant that, once a year, along with other academics with research grants (in my case from STFC) I had to waste hours uploading bibliometric and other data by hand. A sensible system would have harvested this automatically as it is mostly available online at various locations or allowed users simply to upload their own publication list as a file; most of us keep an up-to-date list of publications for various reasons (including vanity!) anyway. Institutions also keep track of all this stuff independently. All this duplication seemed utterly pointless.

I always wondered what happened to the information I uploaded every year, which seemed to disappear without trace into the bowels of RCUK. I assume it was used for something, but mere researchers were never told to what purpose. I guess it was used to assess the performance of researchers in some way.

When I left the UK in 2018 to work full-time in Ireland, I took great pleasure in ignoring the multiple emails demanding that I do yet another Researchfish upload. The automated reminders turned into individual emails threatening that I would never again be eligible for funding if I didn’t do it, to which I eventually replied that I wouldn’t be applying for UK research grants anymore anyway. So there. Eventually the emails stopped.

Then, about three years ago, ResearchFish went from being merely pointless to downright sinister as a scandal erupted about the company that operates it (called Infotech), involving the abuse of data and the bullying of academics. I wrote about this here. It then transpired that UKRI, the umbrella organization governing the UK’s research council had been actively conniving with Infotech to target critics. An inquiry was promised but I don’t know what became of that.

Anyway, all that was a while ago and I neither longer live nor work in the UK so why mention ResearchFish again, now?

The reason is something that shocked me when I found out about it a few days ago. Researchfish is now operated by commercial publishing house Elsevier.

Words fail. I can’t be the only person to see a gigantic conflict of interest. How can a government agency allow the assessment of its research outputs to be outsourced to a company that profits hugely by the publication of those outputs? There’s a phrase in British English which I think is in fairly common usage: marking your own homework. This relates to individuals or organizations who have been given the responsibility for regulating their own products. Is very apt here.

The acquisition of Researchfish isn’t the only example of Elsevier getting its talons stuck into academia life. Elsevier also “runs” the bibliometric service Scopus which it markets as a sort of quality indicator for academic articles. I put “runs” in inverted commas because Scopus is hopelessly inaccurate and unreliable. I can certainly speak from experience on that. Nevertheless, Elsevier has managed to dupe research managers – clearly not the brightest people in the world – into thinking that Scopus is a quality product. I suppose the more you pay for something the less inclined you are to doubt its worth, because if you do find you have paid worthless junk you look like an idiot.

A few days ago I posted a piece that include this excerpt from an article in Wired:

Every industry has certain problems universally acknowledged as broken: insurance in health care, licensing in music, standardized testing in education, tipping in the restaurant business. In academia, it’s publishing. Academic publishing is dominated by for-profit giants like Elsevier and Springer. Calling their practice a form of thuggery isn’t so much an insult as an economic observation. 

With the steady encroachment of the likes of Elsevier into research assessment, it is clear that as well as raking in huge profits, the thugs are now also assuming the role of the police. The academic publishing industry is a monstrous juggernaut that is doing untold damage to research and is set to do more. It has to stop.

#bibliometrics #Elsevier #Infotech #ResearchAssessment #Researchfish #SCOPUS #UKRI

Our monthly newsletter is out!
The latest episode of ‘Connecting the Dots’, on monitoring
#OpenScience at the University level for #ResearchAssessment
Next webinar on 3 April on the new @OpenAIREGraph #API with @vergoulis
Access the materials from the webinar on #GraspOS benefits to Finnish organisations
The role of #OpenScience and #ResearchAssessment in the proposal for the #ERA Agenda 2025-2027
Read it here https://mailchi.mp/1102055a2e5a/graspos-newsletter-12746184
Subscribe graspos.eu/newsletters

@graspos publishes the third edition of "Connecting the Dots: Monitoring Open Science at university level for research assessment ".

Following the GraspOS Community of Practice meeting held on 22 January 2025, Rita Morais from @EUA, Ana Đorđević & Biljana Kosanović from the University of Belgrade, and I, answer questions and share insights on Open Science Monitoring in our respective organisations, and its links with research assessment.

https://graspos.eu/connecting-the-dots-monitoring-open-science-at-university-level-for-research-assessment

#OpenScienceMonitoring
#OpenScience
#ResearchAssessment
#UnivLorraine
#BelgradeUniviersity

cc @osmi

Le Comité Évaluation et science ouverte de l’Académie des sciences (CoÉSO) publie un nouveau rapport intitulé « Évaluation des enseignants-chercheurs, des chercheurs, de leurs équipes et de leurs projets scientifiques dans le contexte de la science ouverte ».

Quatre recommandations :
* Résister aux pressions de l’édition scientifique commerciale
* Simplifier les procédures d’évaluation
* Rendre l’évaluation plus pertinente
* Garantir une qualité des jurys à la hauteur des enjeux

https://www.academie-sciences.fr/nouveau-rapport-evaluation-des-enseignants-chercheurs

#scienceouverte
#CoARA
#ResearchAssessment

An entertaining, informative and overall really well done video about the h-index and why you shouldn't use it, by @stefhaustein, @carey_mlchen et al.

I really will be sharing this video a lot: "What is the h-index and what are its limitations? Or: Stop using the h-index"

https://www.youtube.com/watch?v=HSf79S3XkJw

#hIndex #bibliometrics #researchEvaluation #researchAssessment #publishOrPerish

@academicchatter

Registration is still open! On Monday 24 March, #GraspOS partners CSC - IT Center for Science, Federation of Finnish Learned Societies, and University of Eastern Finland will explore how #GraspOS solutions support #OpenScience-aware Responsible #ResearchAssessment for organisations in Finland

Learn more about the #webinar https://graspos.eu/press/events/national-event/graspos-benefits-to-finnish-organisations
Register now https://cscfi.zoom.us/meeting/register/3v4R00PTRz6LzpiDBOw78g#/registration

The Colombian Ministry of Science published the preliminary results of national #researchassessment exercise. It had many errors and inconsistencies.

I made a post on #linkedin with some criticisms. It went sort of viral among our community and it catched the attention from El Espectador, one of the biggest newspaper. I was one of the interviewed for an article.

It was a honor for me. But the important issue that I want to highlight is that if we, as researchers, want to have a impact on society, we need to speak through the channels where society is. Not Just where we think we should speak

#colombia #ScienceCom #Research

Webinar announcement!

The next EOSC Finnish Forum webinar will take place on 24 March from 12:00 to 13:00 CET and will look at how the solutions developed in #GraspOS advance Open Science-aware responsible research assessment and benefit Finnish organisations.

Learn more: https://graspos.eu/press/events/national-event/graspos-benefits-to-finnish-organisations

Register: https://cscfi.zoom.us/meeting/register/3v4R00PTRz6LzpiDBOw78g#/

#OpenScience #EOSC #ResearchAssessment #ReformingRA

Science thrives when knowledge is open! Check out this strategy for #OpenScience: integrating #FAIR & #CARE principles, responsible researcher assessment #DORA, and digital infrastructure for transparent research
@BarcelonaDORI:

https://doi.org/10.5281/zenodo.15007508

Let's build a transparent, inclusive, and impactful research ecosystem together! Global impact begins locally!

#OpenAccess #FAIRdata #ResearchIntegrity #CitizenScience #ResearchAssessment #SciComm #EOSC #HigherEd

One week to go before the next #GraspOS Community of Practice meeting!
Join us online on 12 March at 10:00 CET to explore the role of context in Responsible #ResearchAssessment and to learn about the diverse assessment approaches observed across the nine #GraspOS pilots

Sign up here to receive the link to join the meeting closer to the event https://graspos.eu/community-of-practice-on-open-science-and-responsible-research-assessment

@tatumcc @cwts @CoARAssessment @OpenAIRE

Today's Open Science Day at @GFZ

Including a talk by @mathijsvleugel on "Rethinking Research Assessment: Moving Beyond Traditional Metrics to Foster Quality and Impact" featuring insights on @DORAssessment and @CoARAssessment

Including further talks as well as hands-on workshop sessions on #OpenAccess #FAIR #ResearchAssessment

https://events.hifis.net/event/2175/timetable/#20250303

#OpenScience #ResearchAssessment #OpenScienceDay (lmf)

The @CoARAssessment Working Group #OI4RRA has published its first report on the principles & framework of open infrastructures for responsible #ResearchAssessment.

Access the report: https://zenodo.org/records/14844582

Read our summary: https://graspos.eu/oi4rra-first-report-published

#OpenScience #ReformingRA #CoARA #OpenInfrastructure
@OpenAIRE

Interested in learning more about the role of Context in Responsible Research Assessment?
Join the next #GraspOS Community of Practice meeting on 12 March, panelists will discuss diverse approaches in #ResearchAssessment, and explore broader implications for the reform
https://graspos.eu/community-of-practice-on-open-science-and-responsible-research-assessment

#OpenScience #CoARA

Today is the 23d birthday of the Budapest Open Access Initiative.
https://www.budapestopenaccessinitiative.org/read/

BOAI is still active and issued its 20th anniversary recommendations in 2022.
https://www.budapestopenaccessinitiative.org/boai20/

Unlike previous BOAI statements, which made many recommendations, the 20th anniversary statement deliberately focused on just a small number of top priorities:

1. Adopting #OpenInfrastructure
2. Reforming #ResearchAssessment
3. Moving away from #APCs
4. Moving away from #ReadAndPublish agreements.

I'm proud of my association with the #BOAI, #BOAI10, and #BOAI20.

Happy #ValentinesDay to all who are working for #OpenAccess worldwide.

Our January newsletter is here! Featuring:

The latest “Cross-project conversations” interview on the @BarcelonaDORI, open research information and research assessment reform with @MsPhelps, @jannepolonen and @vergoulis
A recap of the #EOSC Winter School 2025
Next #GraspOS Webinar on Narrative CVs
Next #GraspOS Community of Practice meeting on the importance of context in #ResearchAssessment
... and more! https://graspos.eu/newsletters

#ReformingRA #OpenScience #CoARA #BarcelonaDORI

This is an important announcement!

Read today OpenAIRE's Action Plan to drive forward and reform Research Assessment!

As we're pushing for a fairer, more transparent approach to evaluating #research this Action Plan aligns with the purposes and
@CoARAssessment ’s commitments. Read all the details for this major step towards academic reform.

Check the full article: https://shorturl.at/bs2PV

#ResearchAssessment #COARA #OpenScience #AcademicReform